2023-02-13 11:52:58,458 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-02-13 11:52:58,706 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed 2023-02-13 11:52:58,725 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-02-13 11:52:58,725 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307, deleteOnExit=true 2023-02-13 11:52:58,726 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-02-13 11:52:58,727 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/test.cache.data in system properties and HBase conf 2023-02-13 11:52:58,727 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/hadoop.tmp.dir in system properties and HBase conf 2023-02-13 11:52:58,728 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/hadoop.log.dir in system properties and HBase conf 2023-02-13 11:52:58,729 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/mapreduce.cluster.local.dir in system properties and HBase conf 2023-02-13 11:52:58,729 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-02-13 11:52:58,730 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-02-13 11:52:58,838 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-02-13 11:52:59,206 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-02-13 11:52:59,210 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-02-13 11:52:59,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-02-13 11:52:59,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-02-13 11:52:59,211 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 11:52:59,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-02-13 11:52:59,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-02-13 11:52:59,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-02-13 11:52:59,212 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 11:52:59,213 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-02-13 11:52:59,213 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/nfs.dump.dir in system properties and HBase conf 2023-02-13 11:52:59,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/java.io.tmpdir in system properties and HBase conf 2023-02-13 11:52:59,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/dfs.journalnode.edits.dir in system properties and HBase conf 2023-02-13 11:52:59,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-02-13 11:52:59,214 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-02-13 11:52:59,686 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 11:52:59,690 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 11:53:00,159 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-02-13 11:53:00,356 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-02-13 11:53:00,377 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 11:53:00,425 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 11:53:00,458 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/java.io.tmpdir/Jetty_localhost_localdomain_43827_hdfs____p97e60/webapp 2023-02-13 11:53:00,633 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:43827 2023-02-13 11:53:00,644 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-02-13 11:53:00,644 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-02-13 11:53:01,501 WARN [Listener at localhost.localdomain/44261] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 11:53:01,583 WARN [Listener at localhost.localdomain/44261] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 11:53:01,601 WARN [Listener at localhost.localdomain/44261] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 11:53:01,607 INFO [Listener at localhost.localdomain/44261] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 11:53:01,612 INFO [Listener at localhost.localdomain/44261] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/java.io.tmpdir/Jetty_localhost_33985_datanode____rmxldp/webapp 2023-02-13 11:53:01,694 INFO [Listener at localhost.localdomain/44261] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33985 2023-02-13 11:53:01,969 WARN [Listener at localhost.localdomain/41351] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 11:53:01,980 WARN [Listener at localhost.localdomain/41351] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 11:53:01,985 WARN [Listener at localhost.localdomain/41351] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 11:53:01,987 INFO [Listener at localhost.localdomain/41351] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 11:53:01,993 INFO [Listener at localhost.localdomain/41351] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/java.io.tmpdir/Jetty_localhost_34357_datanode____r1h4aj/webapp 2023-02-13 11:53:02,086 INFO [Listener at localhost.localdomain/41351] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34357 2023-02-13 11:53:02,104 WARN [Listener at localhost.localdomain/46381] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 11:53:02,131 WARN [Listener at localhost.localdomain/46381] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-02-13 11:53:02,135 WARN [Listener at localhost.localdomain/46381] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-02-13 11:53:02,137 INFO [Listener at localhost.localdomain/46381] log.Slf4jLog(67): jetty-6.1.26 2023-02-13 11:53:02,146 INFO [Listener at localhost.localdomain/46381] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/java.io.tmpdir/Jetty_localhost_42927_datanode____rw45bb/webapp 2023-02-13 11:53:02,223 INFO [Listener at localhost.localdomain/46381] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42927 2023-02-13 11:53:02,232 WARN [Listener at localhost.localdomain/45781] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-02-13 11:53:03,169 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xebc455de9ca00344: Processing first storage report for DS-9527f853-c493-4a33-95f9-b6d54e359e57 from datanode a211374e-09f0-4304-98b9-bf41d73f5c15 2023-02-13 11:53:03,171 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xebc455de9ca00344: from storage DS-9527f853-c493-4a33-95f9-b6d54e359e57 node DatanodeRegistration(127.0.0.1:33085, datanodeUuid=a211374e-09f0-4304-98b9-bf41d73f5c15, infoPort=41529, infoSecurePort=0, ipcPort=46381, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,171 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3a6619a90272552f: Processing first storage report for DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8 from datanode 1e23576d-1428-4e1d-8a86-c9c4402d4f5d 2023-02-13 11:53:03,171 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3a6619a90272552f: from storage DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8 node DatanodeRegistration(127.0.0.1:44945, datanodeUuid=1e23576d-1428-4e1d-8a86-c9c4402d4f5d, infoPort=34511, infoSecurePort=0, ipcPort=45781, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,171 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc7d58132abd56a27: Processing first storage report for DS-dace5762-734d-49cb-bf69-1a4a9f441ab5 from datanode 94b0e9f0-f834-4d10-b805-49b63beb355b 2023-02-13 11:53:03,171 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc7d58132abd56a27: from storage DS-dace5762-734d-49cb-bf69-1a4a9f441ab5 node DatanodeRegistration(127.0.0.1:41815, datanodeUuid=94b0e9f0-f834-4d10-b805-49b63beb355b, infoPort=46145, infoSecurePort=0, ipcPort=41351, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xebc455de9ca00344: Processing first storage report for DS-1dfa29b4-dd1f-407b-843d-c5662ec007e0 from datanode a211374e-09f0-4304-98b9-bf41d73f5c15 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xebc455de9ca00344: from storage DS-1dfa29b4-dd1f-407b-843d-c5662ec007e0 node DatanodeRegistration(127.0.0.1:33085, datanodeUuid=a211374e-09f0-4304-98b9-bf41d73f5c15, infoPort=41529, infoSecurePort=0, ipcPort=46381, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x3a6619a90272552f: Processing first storage report for DS-0b7cb5cb-acd7-44ae-b147-265bed8f22f9 from datanode 1e23576d-1428-4e1d-8a86-c9c4402d4f5d 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x3a6619a90272552f: from storage DS-0b7cb5cb-acd7-44ae-b147-265bed8f22f9 node DatanodeRegistration(127.0.0.1:44945, datanodeUuid=1e23576d-1428-4e1d-8a86-c9c4402d4f5d, infoPort=34511, infoSecurePort=0, ipcPort=45781, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xc7d58132abd56a27: Processing first storage report for DS-5c77c6fa-e2a0-4b2d-8493-9ea3dc732b85 from datanode 94b0e9f0-f834-4d10-b805-49b63beb355b 2023-02-13 11:53:03,172 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xc7d58132abd56a27: from storage DS-5c77c6fa-e2a0-4b2d-8493-9ea3dc732b85 node DatanodeRegistration(127.0.0.1:41815, datanodeUuid=94b0e9f0-f834-4d10-b805-49b63beb355b, infoPort=46145, infoSecurePort=0, ipcPort=41351, storageInfo=lv=-57;cid=testClusterID;nsid=609781459;c=1676289179769), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-02-13 11:53:03,256 DEBUG [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed 2023-02-13 11:53:03,303 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/zookeeper_0, clientPort=55638, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-02-13 11:53:03,313 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=55638 2023-02-13 11:53:03,321 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:03,324 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:03,954 INFO [Listener at localhost.localdomain/45781] util.FSUtils(479): Created version file at hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567 with version=8 2023-02-13 11:53:03,955 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/hbase-staging 2023-02-13 11:53:04,206 INFO [Listener at localhost.localdomain/45781] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-02-13 11:53:04,570 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): master/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 11:53:04,595 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:04,595 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:04,596 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 11:53:04,596 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:04,596 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 11:53:04,708 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 11:53:04,765 DEBUG [Listener at localhost.localdomain/45781] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-02-13 11:53:04,836 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /188.40.62.62:44445 2023-02-13 11:53:04,845 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:04,846 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:04,862 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44445 connecting to ZooKeeper ensemble=127.0.0.1:55638 2023-02-13 11:53:04,956 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:444450x0, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 11:53:04,960 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): master:44445-0x10052bc7cb00000 connected 2023-02-13 11:53:05,039 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 11:53:05,041 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:53:05,047 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 11:53:05,057 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44445 2023-02-13 11:53:05,057 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44445 2023-02-13 11:53:05,058 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44445 2023-02-13 11:53:05,058 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44445 2023-02-13 11:53:05,058 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44445 2023-02-13 11:53:05,063 INFO [Listener at localhost.localdomain/45781] master.HMaster(439): hbase.rootdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567, hbase.cluster.distributed=false 2023-02-13 11:53:05,122 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 11:53:05,122 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,122 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,122 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 11:53:05,123 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,123 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 11:53:05,126 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 11:53:05,129 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /188.40.62.62:43159 2023-02-13 11:53:05,131 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 11:53:05,136 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 11:53:05,137 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,139 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,141 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:43159 connecting to ZooKeeper ensemble=127.0.0.1:55638 2023-02-13 11:53:05,153 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:431590x0, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 11:53:05,154 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:43159-0x10052bc7cb00001 connected 2023-02-13 11:53:05,154 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 11:53:05,156 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:53:05,157 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 11:53:05,157 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=43159 2023-02-13 11:53:05,157 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=43159 2023-02-13 11:53:05,158 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=43159 2023-02-13 11:53:05,159 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=43159 2023-02-13 11:53:05,159 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=43159 2023-02-13 11:53:05,171 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 11:53:05,171 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,171 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,172 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 11:53:05,172 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,172 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 11:53:05,173 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 11:53:05,174 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /188.40.62.62:44985 2023-02-13 11:53:05,175 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 11:53:05,175 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 11:53:05,176 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,179 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,180 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44985 connecting to ZooKeeper ensemble=127.0.0.1:55638 2023-02-13 11:53:05,191 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:449850x0, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 11:53:05,193 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44985-0x10052bc7cb00002 connected 2023-02-13 11:53:05,193 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 11:53:05,194 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:53:05,195 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 11:53:05,196 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44985 2023-02-13 11:53:05,196 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44985 2023-02-13 11:53:05,196 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44985 2023-02-13 11:53:05,197 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44985 2023-02-13 11:53:05,197 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44985 2023-02-13 11:53:05,213 INFO [Listener at localhost.localdomain/45781] client.ConnectionUtils(127): regionserver/jenkins-hbase16:0 server-side Connection retries=45 2023-02-13 11:53:05,213 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,214 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,214 INFO [Listener at localhost.localdomain/45781] ipc.RWQueueRpcExecutor(105): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-02-13 11:53:05,214 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-02-13 11:53:05,214 INFO [Listener at localhost.localdomain/45781] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-02-13 11:53:05,214 INFO [Listener at localhost.localdomain/45781] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-02-13 11:53:05,215 INFO [Listener at localhost.localdomain/45781] ipc.NettyRpcServer(120): Bind to /188.40.62.62:44743 2023-02-13 11:53:05,216 INFO [Listener at localhost.localdomain/45781] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-02-13 11:53:05,217 DEBUG [Listener at localhost.localdomain/45781] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-02-13 11:53:05,218 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,220 INFO [Listener at localhost.localdomain/45781] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,221 INFO [Listener at localhost.localdomain/45781] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:44743 connecting to ZooKeeper ensemble=127.0.0.1:55638 2023-02-13 11:53:05,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:447430x0, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-02-13 11:53:05,234 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:447430x0, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 11:53:05,234 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(623): regionserver:44743-0x10052bc7cb00003 connected 2023-02-13 11:53:05,235 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:53:05,236 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-02-13 11:53:05,236 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44743 2023-02-13 11:53:05,236 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44743 2023-02-13 11:53:05,237 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44743 2023-02-13 11:53:05,237 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44743 2023-02-13 11:53:05,238 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44743 2023-02-13 11:53:05,242 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2158): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:05,262 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 11:53:05,264 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:05,291 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 11:53:05,291 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 11:53:05,291 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 11:53:05,291 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-02-13 11:53:05,292 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:05,293 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 11:53:05,294 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(224): Deleting ZNode for /hbase/backup-masters/jenkins-hbase16.apache.org,44445,1676289184060 from backup master directory 2023-02-13 11:53:05,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-02-13 11:53:05,303 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:05,303 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-02-13 11:53:05,303 WARN [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 11:53:05,304 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ActiveMasterManager(234): Registered as active master=jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:05,307 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-02-13 11:53:05,308 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-02-13 11:53:05,397 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] util.FSUtils(628): Created cluster ID file at hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/hbase.id with ID: 350c8f39-7298-4a78-bf67-96283c9246d2 2023-02-13 11:53:05,440 INFO [master/jenkins-hbase16:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-02-13 11:53:05,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:05,511 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x1482fe3e to 127.0.0.1:55638 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 11:53:05,550 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@21261550, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 11:53:05,570 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 11:53:05,572 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-02-13 11:53:05,585 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-02-13 11:53:05,585 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-02-13 11:53:05,587 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:53:05,591 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:849) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2178) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:523) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:53:05,592 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 11:53:05,620 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7689): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store-tmp 2023-02-13 11:53:05,656 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:05,656 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 11:53:05,657 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:53:05,657 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:53:05,657 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 11:53:05,657 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:53:05,657 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:53:05,657 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676289185656Disabling compacts and flushes for region at 1676289185656Disabling writes for close at 1676289185657 (+1 ms)Writing region close event to WAL at 1676289185657Closed at 1676289185657 2023-02-13 11:53:05,659 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/WALs/jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:05,677 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C44445%2C1676289184060, suffix=, logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/WALs/jenkins-hbase16.apache.org,44445,1676289184060, archiveDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/oldWALs, maxLogs=10 2023-02-13 11:53:05,724 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK] 2023-02-13 11:53:05,724 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK] 2023-02-13 11:53:05,724 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK] 2023-02-13 11:53:05,731 DEBUG [RS-EventLoopGroup-5-1] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:53:05,794 INFO [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/WALs/jenkins-hbase16.apache.org,44445,1676289184060/jenkins-hbase16.apache.org%2C44445%2C1676289184060.1676289185685 2023-02-13 11:53:05,795 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK], DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK], DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK]] 2023-02-13 11:53:05,795 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7850): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:05,796 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(865): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:05,798 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7890): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,799 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(7893): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,847 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,853 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-02-13 11:53:05,872 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-02-13 11:53:05,882 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:05,888 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,890 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,903 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1054): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-02-13 11:53:05,906 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:05,907 INFO [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(1071): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61902414, jitterRate=-0.07758215069770813}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 11:53:05,908 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] regionserver.HRegion(964): Region open journal for 1595e783b53d99cd5eef43b6debb2682: Writing region info on filesystem at 1676289185820Initializing all the Stores at 1676289185822 (+2 ms)Instantiating store for column family {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289185823 (+1 ms)Cleaning up temporary data from old regions at 1676289185895 (+72 ms)Cleaning up detritus from prior splits at 1676289185896 (+1 ms)Region opened successfully at 1676289185908 (+12 ms) 2023-02-13 11:53:05,912 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-02-13 11:53:05,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:05,930 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:05,932 INFO [master/jenkins-hbase16:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:05,933 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:05,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 26 msec 2023-02-13 11:53:05,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-02-13 11:53:05,981 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-02-13 11:53:05,986 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-02-13 11:53:06,005 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-02-13 11:53:06,008 INFO [master/jenkins-hbase16:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-02-13 11:53:06,009 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-02-13 11:53:06,013 INFO [master/jenkins-hbase16:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-02-13 11:53:06,017 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-02-13 11:53:06,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:06,052 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-02-13 11:53:06,052 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-02-13 11:53:06,062 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-02-13 11:53:06,075 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 11:53:06,075 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 11:53:06,075 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 11:53:06,075 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-02-13 11:53:06,075 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:06,076 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(739): Active/primary master=jenkins-hbase16.apache.org,44445,1676289184060, sessionid=0x10052bc7cb00000, setting cluster-up flag (Was=false) 2023-02-13 11:53:06,108 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:06,138 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-02-13 11:53:06,139 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:06,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:06,186 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-02-13 11:53:06,187 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:06,190 WARN [master/jenkins-hbase16:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.hbase-snapshot/.tmp 2023-02-13 11:53:06,244 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(952): ClusterId : 350c8f39-7298-4a78-bf67-96283c9246d2 2023-02-13 11:53:06,244 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(952): ClusterId : 350c8f39-7298-4a78-bf67-96283c9246d2 2023-02-13 11:53:06,244 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(952): ClusterId : 350c8f39-7298-4a78-bf67-96283c9246d2 2023-02-13 11:53:06,249 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 11:53:06,249 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 11:53:06,249 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-02-13 11:53:06,271 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 11:53:06,271 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 11:53:06,271 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 11:53:06,271 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-02-13 11:53:06,271 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 11:53:06,271 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-02-13 11:53:06,284 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:06,287 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 11:53:06,287 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 11:53:06,287 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-02-13 11:53:06,289 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ReadOnlyZKClient(139): Connect 0x175830b3 to 127.0.0.1:55638 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 11:53:06,289 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ReadOnlyZKClient(139): Connect 0x27b5e257 to 127.0.0.1:55638 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 11:53:06,290 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ReadOnlyZKClient(139): Connect 0x0e28de38 to 127.0.0.1:55638 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 11:53:06,319 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 11:53:06,319 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 11:53:06,319 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 11:53:06,319 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=5, maxPoolSize=5 2023-02-13 11:53:06,319 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase16:0, corePoolSize=10, maxPoolSize=10 2023-02-13 11:53:06,320 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,320 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 11:53:06,320 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,321 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289216321 2023-02-13 11:53:06,323 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-02-13 11:53:06,328 DEBUG [RS:2;jenkins-hbase16:44743] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@297218d1, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 11:53:06,329 DEBUG [RS:0;jenkins-hbase16:43159] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@ef318d, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 11:53:06,329 DEBUG [RS:2;jenkins-hbase16:44743] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@2d4d8927, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 11:53:06,328 DEBUG [RS:1;jenkins-hbase16:44985] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@199fcc28, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 11:53:06,329 DEBUG [RS:0;jenkins-hbase16:43159] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d0b548, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 11:53:06,329 DEBUG [RS:1;jenkins-hbase16:44985] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1da05bc3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 11:53:06,329 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:06,330 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-02-13 11:53:06,335 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 11:53:06,336 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-02-13 11:53:06,343 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-02-13 11:53:06,343 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-02-13 11:53:06,344 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-02-13 11:53:06,344 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-02-13 11:53:06,344 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,346 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-02-13 11:53:06,347 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-02-13 11:53:06,347 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-02-13 11:53:06,350 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-02-13 11:53:06,350 INFO [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-02-13 11:53:06,352 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676289186352,5,FailOnTimeoutGroup] 2023-02-13 11:53:06,352 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676289186352,5,FailOnTimeoutGroup] 2023-02-13 11:53:06,352 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,352 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1451): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-02-13 11:53:06,353 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,354 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,359 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase16:43159 2023-02-13 11:53:06,360 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase16:44743 2023-02-13 11:53:06,360 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase16:44985 2023-02-13 11:53:06,370 INFO [RS:2;jenkins-hbase16:44743] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 11:53:06,371 INFO [RS:2;jenkins-hbase16:44743] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 11:53:06,370 INFO [RS:0;jenkins-hbase16:43159] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 11:53:06,371 INFO [RS:0;jenkins-hbase16:43159] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 11:53:06,370 INFO [RS:1;jenkins-hbase16:44985] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-02-13 11:53:06,371 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 11:53:06,371 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 11:53:06,371 INFO [RS:1;jenkins-hbase16:44985] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-02-13 11:53:06,372 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1023): About to register with Master. 2023-02-13 11:53:06,377 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,44445,1676289184060 with isa=jenkins-hbase16.apache.org/188.40.62.62:44743, startcode=1676289185212 2023-02-13 11:53:06,377 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,44445,1676289184060 with isa=jenkins-hbase16.apache.org/188.40.62.62:43159, startcode=1676289185121 2023-02-13 11:53:06,377 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(2810): reportForDuty to master=jenkins-hbase16.apache.org,44445,1676289184060 with isa=jenkins-hbase16.apache.org/188.40.62.62:44985, startcode=1676289185170 2023-02-13 11:53:06,379 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:06,380 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:06,381 INFO [PEWorker-1] regionserver.HRegion(7671): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567 2023-02-13 11:53:06,400 DEBUG [RS:0;jenkins-hbase16:43159] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 11:53:06,400 DEBUG [RS:2;jenkins-hbase16:44743] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 11:53:06,400 DEBUG [RS:1;jenkins-hbase16:44985] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-02-13 11:53:06,408 DEBUG [PEWorker-1] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:06,412 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 11:53:06,415 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/info 2023-02-13 11:53:06,416 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 11:53:06,417 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,417 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 11:53:06,420 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/rep_barrier 2023-02-13 11:53:06,421 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 11:53:06,421 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,422 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 11:53:06,424 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/table 2023-02-13 11:53:06,425 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 11:53:06,426 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,428 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740 2023-02-13 11:53:06,430 DEBUG [PEWorker-1] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740 2023-02-13 11:53:06,437 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 11:53:06,439 DEBUG [PEWorker-1] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 11:53:06,445 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:06,445 INFO [PEWorker-1] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70759339, jitterRate=0.05439631640911102}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 11:53:06,446 DEBUG [PEWorker-1] regionserver.HRegion(964): Region open journal for 1588230740: Writing region info on filesystem at 1676289186409Initializing all the Stores at 1676289186411 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676289186411Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289186411Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676289186411Cleaning up temporary data from old regions at 1676289186434 (+23 ms)Cleaning up detritus from prior splits at 1676289186435 (+1 ms)Region opened successfully at 1676289186446 (+11 ms) 2023-02-13 11:53:06,446 DEBUG [PEWorker-1] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 11:53:06,446 INFO [PEWorker-1] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 11:53:06,446 DEBUG [PEWorker-1] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 11:53:06,446 DEBUG [PEWorker-1] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 11:53:06,446 DEBUG [PEWorker-1] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 11:53:06,447 INFO [PEWorker-1] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 11:53:06,447 DEBUG [PEWorker-1] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676289186446Disabling compacts and flushes for region at 1676289186446Disabling writes for close at 1676289186446Writing region close event to WAL at 1676289186447 (+1 ms)Closed at 1676289186447 2023-02-13 11:53:06,452 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:06,452 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-02-13 11:53:06,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-02-13 11:53:06,460 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:59801, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 11:53:06,460 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:35903, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 11:53:06,460 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44945, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-02-13 11:53:06,472 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=44445] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,473 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.ServerManager(394): Registering regionserver=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,476 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-02-13 11:53:06,478 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 11:53:06,489 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567 2023-02-13 11:53:06,489 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567 2023-02-13 11:53:06,489 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1596): Config from master: hbase.rootdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567 2023-02-13 11:53:06,490 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44261 2023-02-13 11:53:06,490 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44261 2023-02-13 11:53:06,490 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 11:53:06,490 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 11:53:06,490 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1596): Config from master: fs.defaultFS=hdfs://localhost.localdomain:44261 2023-02-13 11:53:06,490 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1596): Config from master: hbase.master.info.port=-1 2023-02-13 11:53:06,549 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:53:06,561 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ZKUtil(162): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,561 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ZKUtil(162): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,562 WARN [RS:2;jenkins-hbase16:44743] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 11:53:06,563 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ZKUtil(162): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,563 INFO [RS:2;jenkins-hbase16:44743] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 11:53:06,562 WARN [RS:1;jenkins-hbase16:44985] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 11:53:06,563 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,44743,1676289185212] 2023-02-13 11:53:06,564 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,44985,1676289185170] 2023-02-13 11:53:06,564 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase16.apache.org,43159,1676289185121] 2023-02-13 11:53:06,563 WARN [RS:0;jenkins-hbase16:43159] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-02-13 11:53:06,563 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,563 INFO [RS:1;jenkins-hbase16:44985] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 11:53:06,564 INFO [RS:0;jenkins-hbase16:43159] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 11:53:06,565 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,565 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1947): logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,582 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ZKUtil(162): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,582 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ZKUtil(162): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,582 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ZKUtil(162): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,582 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ZKUtil(162): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,582 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ZKUtil(162): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,582 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ZKUtil(162): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,583 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ZKUtil(162): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,583 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ZKUtil(162): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,583 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ZKUtil(162): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,591 DEBUG [RS:1;jenkins-hbase16:44985] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 11:53:06,591 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 11:53:06,591 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-02-13 11:53:06,598 INFO [RS:1;jenkins-hbase16:44985] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 11:53:06,598 INFO [RS:2;jenkins-hbase16:44743] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 11:53:06,598 INFO [RS:0;jenkins-hbase16:43159] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-02-13 11:53:06,616 INFO [RS:0;jenkins-hbase16:43159] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 11:53:06,616 INFO [RS:2;jenkins-hbase16:44743] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 11:53:06,616 INFO [RS:1;jenkins-hbase16:44985] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-02-13 11:53:06,620 INFO [RS:1;jenkins-hbase16:44985] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 11:53:06,620 INFO [RS:2;jenkins-hbase16:44743] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 11:53:06,621 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,621 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,620 INFO [RS:0;jenkins-hbase16:43159] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-02-13 11:53:06,621 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,622 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 11:53:06,622 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 11:53:06,622 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer$CompactionChecker(1838): CompactionChecker runs every PT1S 2023-02-13 11:53:06,628 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,628 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,628 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,629 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,629 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,629 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,629 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,629 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,629 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,631 DEBUG [RS:2;jenkins-hbase16:44743] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,631 DEBUG [RS:1;jenkins-hbase16:44985] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,630 DEBUG [jenkins-hbase16:44445] assignment.AssignmentManager(2178): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-02-13 11:53:06,630 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,631 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,631 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,631 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase16:0, corePoolSize=2, maxPoolSize=2 2023-02-13 11:53:06,632 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,632 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,632 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,632 DEBUG [RS:0;jenkins-hbase16:43159] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase16:0, corePoolSize=1, maxPoolSize=1 2023-02-13 11:53:06,635 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,635 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:06,635 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,635 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,636 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,636 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,636 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,636 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,637 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,637 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,643 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:06,643 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:06,643 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:06,644 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:06,647 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,43159,1676289185121, state=OPENING 2023-02-13 11:53:06,653 INFO [RS:1;jenkins-hbase16:44985] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 11:53:06,653 INFO [RS:2;jenkins-hbase16:44743] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 11:53:06,653 INFO [RS:0;jenkins-hbase16:43159] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-02-13 11:53:06,656 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,43159,1676289185121-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,656 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44743,1676289185212-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,656 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44985,1676289185170-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:06,661 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-02-13 11:53:06,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:06,670 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 11:53:06,674 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:06,676 INFO [RS:0;jenkins-hbase16:43159] regionserver.Replication(203): jenkins-hbase16.apache.org,43159,1676289185121 started 2023-02-13 11:53:06,676 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,43159,1676289185121, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:43159, sessionid=0x10052bc7cb00001 2023-02-13 11:53:06,678 INFO [RS:1;jenkins-hbase16:44985] regionserver.Replication(203): jenkins-hbase16.apache.org,44985,1676289185170 started 2023-02-13 11:53:06,678 INFO [RS:2;jenkins-hbase16:44743] regionserver.Replication(203): jenkins-hbase16.apache.org,44743,1676289185212 started 2023-02-13 11:53:06,678 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 11:53:06,678 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,44743,1676289185212, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:44743, sessionid=0x10052bc7cb00003 2023-02-13 11:53:06,678 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1638): Serving as jenkins-hbase16.apache.org,44985,1676289185170, RpcServer on jenkins-hbase16.apache.org/188.40.62.62:44985, sessionid=0x10052bc7cb00002 2023-02-13 11:53:06,678 DEBUG [RS:0;jenkins-hbase16:43159] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,678 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 11:53:06,678 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,43159,1676289185121' 2023-02-13 11:53:06,678 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-02-13 11:53:06,679 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:06,678 DEBUG [RS:1;jenkins-hbase16:44985] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,679 DEBUG [RS:2;jenkins-hbase16:44743] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,680 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44743,1676289185212' 2023-02-13 11:53:06,680 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:06,679 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44985,1676289185170' 2023-02-13 11:53:06,680 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:06,680 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:06,681 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:06,681 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 11:53:06,681 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 11:53:06,681 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 11:53:06,681 DEBUG [RS:0;jenkins-hbase16:43159] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,681 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 11:53:06,681 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,43159,1676289185121' 2023-02-13 11:53:06,682 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:06,681 DEBUG [RS:1;jenkins-hbase16:44985] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:06,682 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 11:53:06,682 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44985,1676289185170' 2023-02-13 11:53:06,682 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-02-13 11:53:06,682 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 11:53:06,683 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-02-13 11:53:06,683 DEBUG [RS:2;jenkins-hbase16:44743] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:06,683 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase16.apache.org,44743,1676289185212' 2023-02-13 11:53:06,683 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-02-13 11:53:06,683 DEBUG [RS:0;jenkins-hbase16:43159] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 11:53:06,683 DEBUG [RS:1;jenkins-hbase16:44985] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 11:53:06,683 DEBUG [RS:2;jenkins-hbase16:44743] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-02-13 11:53:06,684 DEBUG [RS:1;jenkins-hbase16:44985] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 11:53:06,684 DEBUG [RS:0;jenkins-hbase16:43159] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 11:53:06,684 INFO [RS:1;jenkins-hbase16:44985] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 11:53:06,684 INFO [RS:0;jenkins-hbase16:43159] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 11:53:06,685 INFO [RS:1;jenkins-hbase16:44985] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 11:53:06,685 INFO [RS:0;jenkins-hbase16:43159] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 11:53:06,687 DEBUG [RS:2;jenkins-hbase16:44743] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-02-13 11:53:06,687 INFO [RS:2;jenkins-hbase16:44743] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-02-13 11:53:06,687 INFO [RS:2;jenkins-hbase16:44743] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-02-13 11:53:06,800 INFO [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C43159%2C1676289185121, suffix=, logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,43159,1676289185121, archiveDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs, maxLogs=32 2023-02-13 11:53:06,800 INFO [RS:1;jenkins-hbase16:44985] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C44985%2C1676289185170, suffix=, logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44985,1676289185170, archiveDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs, maxLogs=32 2023-02-13 11:53:06,800 INFO [RS:2;jenkins-hbase16:44743] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C44743%2C1676289185212, suffix=, logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44743,1676289185212, archiveDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs, maxLogs=32 2023-02-13 11:53:06,825 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK] 2023-02-13 11:53:06,825 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK] 2023-02-13 11:53:06,825 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK] 2023-02-13 11:53:06,825 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK] 2023-02-13 11:53:06,825 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK] 2023-02-13 11:53:06,826 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK] 2023-02-13 11:53:06,832 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK] 2023-02-13 11:53:06,832 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK] 2023-02-13 11:53:06,832 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK] 2023-02-13 11:53:06,834 INFO [RS:2;jenkins-hbase16:44743] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44743,1676289185212/jenkins-hbase16.apache.org%2C44743%2C1676289185212.1676289186803 2023-02-13 11:53:06,834 DEBUG [RS:2;jenkins-hbase16:44743] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK], DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK], DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK]] 2023-02-13 11:53:06,839 INFO [RS:1;jenkins-hbase16:44985] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,44985,1676289185170/jenkins-hbase16.apache.org%2C44985%2C1676289185170.1676289186803 2023-02-13 11:53:06,839 DEBUG [RS:1;jenkins-hbase16:44985] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK], DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK], DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK]] 2023-02-13 11:53:06,841 INFO [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,43159,1676289185121/jenkins-hbase16.apache.org%2C43159%2C1676289185121.1676289186803 2023-02-13 11:53:06,842 DEBUG [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK], DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK], DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK]] 2023-02-13 11:53:06,864 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:06,866 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 11:53:06,869 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44996, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 11:53:06,881 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:meta,,1.1588230740 2023-02-13 11:53:06,882 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-02-13 11:53:06,886 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(464): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase16.apache.org%2C43159%2C1676289185121.meta, suffix=.meta, logDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,43159,1676289185121, archiveDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs, maxLogs=32 2023-02-13 11:53:06,903 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK] 2023-02-13 11:53:06,906 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK] 2023-02-13 11:53:06,904 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK] 2023-02-13 11:53:06,912 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(758): New WAL /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/WALs/jenkins-hbase16.apache.org,43159,1676289185121/jenkins-hbase16.apache.org%2C43159%2C1676289185121.meta.1676289186887.meta 2023-02-13 11:53:06,912 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] wal.AbstractFSWAL(839): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41815,DS-dace5762-734d-49cb-bf69-1a4a9f441ab5,DISK], DatanodeInfoWithStorage[127.0.0.1:33085,DS-9527f853-c493-4a33-95f9-b6d54e359e57,DISK], DatanodeInfoWithStorage[127.0.0.1:44945,DS-7cc856ae-c4c5-4c5e-ab00-2dad0197c8a8,DISK]] 2023-02-13 11:53:06,912 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:06,917 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:06,917 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-02-13 11:53:06,932 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(8546): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-02-13 11:53:06,934 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-02-13 11:53:06,938 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-02-13 11:53:06,938 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:06,938 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 1588230740 2023-02-13 11:53:06,938 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 1588230740 2023-02-13 11:53:06,941 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-02-13 11:53:06,942 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/info 2023-02-13 11:53:06,942 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/info 2023-02-13 11:53:06,943 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-02-13 11:53:06,943 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,944 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-02-13 11:53:06,945 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/rep_barrier 2023-02-13 11:53:06,945 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/rep_barrier 2023-02-13 11:53:06,945 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-02-13 11:53:06,946 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,946 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-02-13 11:53:06,948 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/table 2023-02-13 11:53:06,948 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/table 2023-02-13 11:53:06,948 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-02-13 11:53:06,949 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:06,951 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740 2023-02-13 11:53:06,955 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740 2023-02-13 11:53:06,961 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-02-13 11:53:06,963 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 1588230740 2023-02-13 11:53:06,965 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59333492, jitterRate=-0.11586207151412964}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-02-13 11:53:06,965 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 1588230740: Running coprocessor pre-open hook at 1676289186938Writing region info on filesystem at 1676289186938Initializing all the Stores at 1676289186940 (+2 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676289186940Instantiating store for column family {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289186940Instantiating store for column family {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676289186940Cleaning up temporary data from old regions at 1676289186956 (+16 ms)Cleaning up detritus from prior splits at 1676289186958 (+2 ms)Running coprocessor post-open hooks at 1676289186965 (+7 ms)Region opened successfully at 1676289186965 2023-02-13 11:53:06,974 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1676289186857 2023-02-13 11:53:06,989 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:meta,,1.1588230740 2023-02-13 11:53:06,990 INFO [RS_OPEN_META-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:meta,,1.1588230740 2023-02-13 11:53:06,990 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase16.apache.org,43159,1676289185121, state=OPEN 2023-02-13 11:53:07,002 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-02-13 11:53:07,003 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-02-13 11:53:07,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-02-13 11:53:07,008 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase16.apache.org,43159,1676289185121 in 328 msec 2023-02-13 11:53:07,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-02-13 11:53:07,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 551 msec 2023-02-13 11:53:07,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 790 msec 2023-02-13 11:53:07,021 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(946): Master startup: status=Wait for region servers to report in, state=RUNNING, startTime=1676289185280, completionTime=-1 2023-02-13 11:53:07,021 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-02-13 11:53:07,021 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:07,079 DEBUG [hconnection-0x61151f3a-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 11:53:07,082 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44998, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 11:53:07,098 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:07,098 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289247098 2023-02-13 11:53:07,098 INFO [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289307098 2023-02-13 11:53:07,099 INFO [master/jenkins-hbase16:0:becomeActiveMaster] assignment.AssignmentManager(1538): Joined the cluster in 77 msec 2023-02-13 11:53:07,145 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44445,1676289184060-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:07,145 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44445,1676289184060-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:07,145 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44445,1676289184060-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:07,147 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase16:44445, period=300000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:07,148 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-02-13 11:53:07,155 DEBUG [master/jenkins-hbase16:0.Chore.1] janitor.CatalogJanitor(175): 2023-02-13 11:53:07,163 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-02-13 11:53:07,166 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(2138): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-02-13 11:53:07,174 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:07,176 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:07,183 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:07,207 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,209 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7 empty. 2023-02-13 11:53:07,210 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,210 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-02-13 11:53:07,252 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:07,254 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7671): creating {ENCODED => e1c6db54e2dd7d1f929d02a4343c29a7, NAME => 'hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:07,273 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:07,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1603): Closing e1c6db54e2dd7d1f929d02a4343c29a7, disabling compactions & flushes 2023-02-13 11:53:07,274 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. after waiting 0 ms 2023-02-13 11:53:07,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,274 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,274 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1557): Region close journal for e1c6db54e2dd7d1f929d02a4343c29a7: Waiting for close lock at 1676289187274Disabling compacts and flushes for region at 1676289187274Disabling writes for close at 1676289187274Writing region close event to WAL at 1676289187274Closed at 1676289187274 2023-02-13 11:53:07,280 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:07,294 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676289187282"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289187282"}]},"ts":"1676289187282"} 2023-02-13 11:53:07,316 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:07,319 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:07,325 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289187319"}]},"ts":"1676289187319"} 2023-02-13 11:53:07,330 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-02-13 11:53:07,350 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:07,352 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:07,352 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:07,353 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:07,353 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:07,358 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e1c6db54e2dd7d1f929d02a4343c29a7, ASSIGN}] 2023-02-13 11:53:07,362 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=e1c6db54e2dd7d1f929d02a4343c29a7, ASSIGN 2023-02-13 11:53:07,364 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=e1c6db54e2dd7d1f929d02a4343c29a7, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:53:07,517 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:07,518 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:07,518 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676289187518"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289187518"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289187518"}]},"ts":"1676289187518"} 2023-02-13 11:53:07,524 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure e1c6db54e2dd7d1f929d02a4343c29a7, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:07,681 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:07,682 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 11:53:07,690 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:56120, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 11:53:07,697 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,698 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e1c6db54e2dd7d1f929d02a4343c29a7, NAME => 'hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:07,701 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:07,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:07,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,702 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,707 INFO [StoreOpener-e1c6db54e2dd7d1f929d02a4343c29a7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,709 DEBUG [StoreOpener-e1c6db54e2dd7d1f929d02a4343c29a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/info 2023-02-13 11:53:07,709 DEBUG [StoreOpener-e1c6db54e2dd7d1f929d02a4343c29a7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/info 2023-02-13 11:53:07,709 INFO [StoreOpener-e1c6db54e2dd7d1f929d02a4343c29a7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1c6db54e2dd7d1f929d02a4343c29a7 columnFamilyName info 2023-02-13 11:53:07,710 INFO [StoreOpener-e1c6db54e2dd7d1f929d02a4343c29a7-1] regionserver.HStore(310): Store=e1c6db54e2dd7d1f929d02a4343c29a7/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:07,712 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,714 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,718 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:53:07,722 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:07,723 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e1c6db54e2dd7d1f929d02a4343c29a7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66493082, jitterRate=-0.009175866842269897}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 11:53:07,724 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e1c6db54e2dd7d1f929d02a4343c29a7: Running coprocessor pre-open hook at 1676289187702Writing region info on filesystem at 1676289187702Initializing all the Stores at 1676289187706 (+4 ms)Instantiating store for column family {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} at 1676289187706Cleaning up temporary data from old regions at 1676289187714 (+8 ms)Cleaning up detritus from prior splits at 1676289187715 (+1 ms)Running coprocessor post-open hooks at 1676289187723 (+8 ms)Region opened successfully at 1676289187724 (+1 ms) 2023-02-13 11:53:07,726 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7., pid=6, masterSystemTime=1676289187681 2023-02-13 11:53:07,732 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,732 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:53:07,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:07,735 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1676289187733"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289187733"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289187733"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289187733"}]},"ts":"1676289187733"} 2023-02-13 11:53:07,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-02-13 11:53:07,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure e1c6db54e2dd7d1f929d02a4343c29a7, server=jenkins-hbase16.apache.org,44743,1676289185212 in 214 msec 2023-02-13 11:53:07,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-02-13 11:53:07,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=e1c6db54e2dd7d1f929d02a4343c29a7, ASSIGN in 386 msec 2023-02-13 11:53:07,747 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:07,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289187747"}]},"ts":"1676289187747"} 2023-02-13 11:53:07,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-02-13 11:53:07,786 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-02-13 11:53:07,787 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:07,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 621 msec 2023-02-13 11:53:07,794 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-02-13 11:53:07,794 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:07,814 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 11:53:07,816 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:56132, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 11:53:07,829 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:07,852 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 11:53:07,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 43 msec 2023-02-13 11:53:07,872 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:07,891 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-02-13 11:53:07,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 31 msec 2023-02-13 11:53:07,933 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-02-13 11:53:07,950 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-02-13 11:53:07,950 INFO [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1077): Master has completed initialization 2.646sec 2023-02-13 11:53:07,954 INFO [master/jenkins-hbase16:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-02-13 11:53:07,956 INFO [master/jenkins-hbase16:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-02-13 11:53:07,957 INFO [master/jenkins-hbase16:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-02-13 11:53:07,958 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44445,1676289184060-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-02-13 11:53:07,960 INFO [master/jenkins-hbase16:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase16.apache.org,44445,1676289184060-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-02-13 11:53:07,971 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster] master.HMaster(1166): Balancer post startup initialization complete, took 0 seconds 2023-02-13 11:53:08,055 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(139): Connect 0x1bf6ae7e to 127.0.0.1:55638 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-02-13 11:53:08,067 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@67f3e316, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-02-13 11:53:08,083 DEBUG [hconnection-0x2c6a7f26-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 11:53:08,093 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:45010, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 11:53:08,102 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:53:08,132 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=319, ProcessCount=169, AvailableMemoryMB=6100 2023-02-13 11:53:08,137 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:08,149 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-02-13 11:53:08,153 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:36342, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-02-13 11:53:08,177 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:08,178 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:53:08,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:08,187 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,190 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:08,192 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:08,195 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,197 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 empty. 2023-02-13 11:53:08,199 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,199 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 11:53:08,219 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:08,221 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 7e2e3da3f4589c85870b60cbdc7b2c06, NAME => 'testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:08,240 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:08,240 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1603): Closing 7e2e3da3f4589c85870b60cbdc7b2c06, disabling compactions & flushes 2023-02-13 11:53:08,240 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,240 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,240 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. after waiting 0 ms 2023-02-13 11:53:08,240 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,241 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,241 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 7e2e3da3f4589c85870b60cbdc7b2c06: Waiting for close lock at 1676289188240Disabling compacts and flushes for region at 1676289188240Disabling writes for close at 1676289188240Writing region close event to WAL at 1676289188241 (+1 ms)Closed at 1676289188241 2023-02-13 11:53:08,245 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:08,246 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289188246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289188246"}]},"ts":"1676289188246"} 2023-02-13 11:53:08,249 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:08,250 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:08,250 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289188250"}]},"ts":"1676289188250"} 2023-02-13 11:53:08,252 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-02-13 11:53:08,270 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:08,272 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:08,272 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:08,272 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:08,272 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:08,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, ASSIGN}] 2023-02-13 11:53:08,277 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, ASSIGN 2023-02-13 11:53:08,278 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:08,428 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:08,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7e2e3da3f4589c85870b60cbdc7b2c06, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289188430"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289188430"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289188430"}]},"ts":"1676289188430"} 2023-02-13 11:53:08,437 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 7e2e3da3f4589c85870b60cbdc7b2c06, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:08,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 7e2e3da3f4589c85870b60cbdc7b2c06, NAME => 'testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:08,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:08,606 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:08,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,607 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,609 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,611 DEBUG [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1 2023-02-13 11:53:08,611 DEBUG [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1 2023-02-13 11:53:08,611 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e2e3da3f4589c85870b60cbdc7b2c06 columnFamilyName cf1 2023-02-13 11:53:08,612 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] regionserver.HStore(310): Store=7e2e3da3f4589c85870b60cbdc7b2c06/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:08,612 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,614 DEBUG [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2 2023-02-13 11:53:08,614 DEBUG [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2 2023-02-13 11:53:08,615 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7e2e3da3f4589c85870b60cbdc7b2c06 columnFamilyName cf2 2023-02-13 11:53:08,615 INFO [StoreOpener-7e2e3da3f4589c85870b60cbdc7b2c06-1] regionserver.HStore(310): Store=7e2e3da3f4589c85870b60cbdc7b2c06/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:08,617 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,618 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:08,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:08,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:08,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 7e2e3da3f4589c85870b60cbdc7b2c06; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61898238, jitterRate=-0.07764437794685364}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:08,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 7e2e3da3f4589c85870b60cbdc7b2c06: Running coprocessor pre-open hook at 1676289188607Writing region info on filesystem at 1676289188607Initializing all the Stores at 1676289188608 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289188608Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289188609 (+1 ms)Cleaning up temporary data from old regions at 1676289188619 (+10 ms)Cleaning up detritus from prior splits at 1676289188619Running coprocessor post-open hooks at 1676289188626 (+7 ms)Region opened successfully at 1676289188626 2023-02-13 11:53:08,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06., pid=11, masterSystemTime=1676289188593 2023-02-13 11:53:08,630 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,631 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=7e2e3da3f4589c85870b60cbdc7b2c06, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289188631"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289188631"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289188631"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289188631"}]},"ts":"1676289188631"} 2023-02-13 11:53:08,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-02-13 11:53:08,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 7e2e3da3f4589c85870b60cbdc7b2c06, server=jenkins-hbase16.apache.org,43159,1676289185121 in 198 msec 2023-02-13 11:53:08,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-02-13 11:53:08,642 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, ASSIGN in 367 msec 2023-02-13 11:53:08,643 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:08,644 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289188643"}]},"ts":"1676289188643"} 2023-02-13 11:53:08,646 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-02-13 11:53:08,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:08,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 469 msec 2023-02-13 11:53:08,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:08,724 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,725 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:08,725 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:08,726 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:08,726 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:08,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,758 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,758 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:08,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:08,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:08,759 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,759 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,759 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,760 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,761 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,762 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:08,762 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:08,763 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:08,764 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:08,764 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:08,764 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:08,764 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:08,764 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:08,765 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:08,765 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:08,765 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:08,765 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:08,765 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:08,766 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:08,766 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,766 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:08,766 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:08,768 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:53:08,795 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,795 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,796 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,796 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,796 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:08,796 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:08,857 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/e04618c2739d4282852e3645e745000d 2023-02-13 11:53:08,919 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/0fc1027ce690420fae581963c26cbeba 2023-02-13 11:53:08,929 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/e04618c2739d4282852e3645e745000d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d 2023-02-13 11:53:08,938 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:08,941 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/0fc1027ce690420fae581963c26cbeba as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba 2023-02-13 11:53:08,950 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:08,952 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 184ms, sequenceid=6, compaction requested=false 2023-02-13 11:53:08,954 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:08,956 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:08,956 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:08,956 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:08,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:08,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:08,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:08,966 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,966 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:08,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,967 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:08,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:08,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:08,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:08,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:08,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:08,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:08,969 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:08,970 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:08,970 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b0dcb71[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:08,970 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,001 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,001 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,002 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,002 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:09,002 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,003 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,002 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,003 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,003 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:09,003 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:09,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,003 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:09,020 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,021 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,021 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,027 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,027 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,027 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,027 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,027 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,028 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:09,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:09,034 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:09,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:09,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:09,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,034 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,045 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,044 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,045 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,046 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,047 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,050 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,050 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,051 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,069 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:09,070 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,070 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,069 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,070 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,070 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,070 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,070 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,071 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,071 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,071 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,071 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,071 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,071 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,072 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,072 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:09,072 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:09,173 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:09,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:09,208 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:09,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:09,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,212 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,212 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,212 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:09,212 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:09,213 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,213 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,219 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,220 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,222 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,221 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,222 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,222 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,223 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,223 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,223 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,223 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,224 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:09,225 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,225 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:09,225 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:09,225 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:09,233 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,241 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,243 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,243 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,245 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/810018f574264e0093ef351705f6d615 2023-02-13 11:53:09,272 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/55d1504fa10340b08761b300867a61ca 2023-02-13 11:53:09,281 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/810018f574264e0093ef351705f6d615 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615 2023-02-13 11:53:09,289 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:09,291 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/55d1504fa10340b08761b300867a61ca as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca 2023-02-13 11:53:09,299 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:09,301 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 76ms, sequenceid=19, compaction requested=false 2023-02-13 11:53:09,301 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:09,301 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,301 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:09,301 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:09,301 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,302 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,302 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,310 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:09,313 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5e3e60ec[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:09,313 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:09,313 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,319 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:09,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,320 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:09,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,328 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,336 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,336 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,336 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,336 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,337 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,337 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:09,341 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:09,341 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:09,341 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:09,341 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:09,341 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,341 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,352 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,353 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,353 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,354 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,354 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,356 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,356 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,357 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,387 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,387 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,388 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:09,387 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,386 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,388 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,389 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,388 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:09,389 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:09,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,389 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,389 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,390 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,489 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:09,492 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:09,523 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:09,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:09,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,525 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,525 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,526 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:09,526 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:09,526 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,526 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,541 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,542 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,542 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:09,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,543 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,544 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,544 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,546 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,546 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,546 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,546 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,547 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,547 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,546 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,547 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,547 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,547 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,548 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:09,549 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:09,549 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,549 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:09,549 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:09,558 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,569 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,569 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,570 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,570 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,570 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,571 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/083efa3a87854ee8b4c5330d9da542a1 2023-02-13 11:53:09,595 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/841e7d25e69e432688c388a5c83d9a7f 2023-02-13 11:53:09,608 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/083efa3a87854ee8b4c5330d9da542a1 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1 2023-02-13 11:53:09,617 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:09,618 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/841e7d25e69e432688c388a5c83d9a7f as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f 2023-02-13 11:53:09,627 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:09,628 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 79ms, sequenceid=32, compaction requested=true 2023-02-13 11:53:09,628 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:09,628 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,628 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:09,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:09,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,636 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:09,639 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@216abb3f[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:09,639 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:09,639 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,650 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:09,650 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,651 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:09,650 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:09,651 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:09,664 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,664 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,664 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,675 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,675 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,675 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,675 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,675 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:09,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:09,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:09,699 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:09,699 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:09,699 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:09,699 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:09,699 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,699 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:09,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,712 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,712 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,712 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,711 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,711 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,713 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,713 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,713 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,712 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,714 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,714 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,714 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,715 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,715 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,719 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,720 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,720 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,727 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,728 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:09,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:09,728 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:09,727 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,728 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,728 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,729 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,729 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:09,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:09,829 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:09,831 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:09,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:09,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:09,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,862 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:09,862 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,863 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:09,863 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:09,863 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,864 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,869 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,869 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,869 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:09,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,870 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,871 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,872 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,872 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,872 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,873 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:09,872 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:09,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:09,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:09,873 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,874 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:09,874 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:09,883 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,891 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,892 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,892 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,899 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/1506dbc99dba455e8a16e5d157481631 2023-02-13 11:53:09,924 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/e3ee0f0b47024188b03604aebb88d01a 2023-02-13 11:53:09,932 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/1506dbc99dba455e8a16e5d157481631 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631 2023-02-13 11:53:09,940 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:09,941 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/e3ee0f0b47024188b03604aebb88d01a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a 2023-02-13 11:53:09,949 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:09,950 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 76ms, sequenceid=45, compaction requested=true 2023-02-13 11:53:09,950 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:09,950 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:09,950 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:09,951 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:09,951 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,951 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:09,951 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:09,989 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,989 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:09,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:09,990 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,990 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:09,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:09,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:09,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:09,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:09,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:09,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:09,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:09,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:09,997 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@703334f7[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:09,997 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:09,997 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,018 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,018 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,018 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:10,018 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,019 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,019 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:10,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,019 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:10,018 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,019 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,019 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:10,028 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,028 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,028 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,028 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,044 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,044 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,044 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:10,052 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:10,052 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:10,052 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:10,052 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:10,052 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,053 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,062 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,063 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,063 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,064 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,065 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,065 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,066 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,077 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,077 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,078 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,078 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,078 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:10,079 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:10,078 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,080 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:10,079 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,080 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,180 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:10,183 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:10,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:10,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:10,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,218 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:10,218 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:10,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:10,220 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,220 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,227 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,227 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,227 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,227 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,228 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,228 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,228 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,229 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,230 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,230 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,230 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,230 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,231 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,230 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,231 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,231 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,232 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,232 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,233 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,233 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:10,235 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,235 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:10,235 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,235 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:10,244 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,253 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,253 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,254 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,255 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,256 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,258 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/437ed8d6d87a46b980251f1551e72bb1 2023-02-13 11:53:10,287 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/c3cecfde6e8449889ef537b61bc6b610 2023-02-13 11:53:10,297 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/437ed8d6d87a46b980251f1551e72bb1 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1 2023-02-13 11:53:10,303 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:10,304 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/c3cecfde6e8449889ef537b61bc6b610 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610 2023-02-13 11:53:10,310 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:10,311 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 76ms, sequenceid=58, compaction requested=true 2023-02-13 11:53:10,311 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:10,311 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:10,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:10,312 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:10,312 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,312 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,312 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,319 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,319 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,319 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,320 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:10,322 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@15003fc3[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:10,322 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:10,322 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,328 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,328 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:10,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,328 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:10,336 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,336 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,336 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,344 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,344 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,344 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,345 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,345 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,344 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:10,351 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:10,351 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:10,351 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:10,351 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:10,351 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,351 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,361 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,362 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,362 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,362 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,363 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,363 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,364 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,366 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,367 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,367 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:10,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,398 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:10,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,399 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:10,399 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,399 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,399 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,399 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,400 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,400 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,400 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,400 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,400 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,499 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:10,502 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:10,531 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:10,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:10,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,534 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:10,534 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-02-13 11:53:10,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-02-13 11:53:10,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,550 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,550 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,550 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:10,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,552 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,552 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,553 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,552 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-02-13 11:53:10,552 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,553 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,553 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,555 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,554 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,555 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:10,555 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:10,555 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,555 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. started... 2023-02-13 11:53:10,555 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:10,556 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,561 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,562 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,574 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/5473d4d9e16f4a2e891dd2ef6366401a 2023-02-13 11:53:10,600 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 2023-02-13 11:53:10,609 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/5473d4d9e16f4a2e891dd2ef6366401a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a 2023-02-13 11:53:10,616 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:10,618 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 2023-02-13 11:53:10,626 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:10,627 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 72ms, sequenceid=71, compaction requested=true 2023-02-13 11:53:10,627 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 7e2e3da3f4589c85870b60cbdc7b2c06: 2023-02-13 11:53:10,628 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:10,628 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:10,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:10,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-02-13 11:53:10,629 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-02-13 11:53:10,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,636 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:10,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-02-13 11:53:10,638 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3894148f[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:10,638 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:10,638 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,644 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,644 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,645 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:10,644 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:10,645 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:10,652 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,652 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,652 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,653 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,661 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-02-13 11:53:10,662 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,662 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-02-13 11:53:10,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:53:10,670 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:10,670 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-02-13 11:53:10,671 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:10,671 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:10,671 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,671 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:10,683 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,683 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:10,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:10,683 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,683 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,683 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:10,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,685 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:10,685 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,686 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:10,687 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,688 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,688 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,689 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,699 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-02-13 11:53:10,700 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:10,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,700 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:10,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:10,700 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,701 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,701 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:10,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:10,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:10,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:10,703 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:10,703 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,703 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-02-13 11:53:10,802 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:10,805 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:10,831 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,838 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:10,839 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:10,839 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:10,839 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:10,846 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, UNASSIGN}] 2023-02-13 11:53:10,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, UNASSIGN 2023-02-13 11:53:10,848 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=7e2e3da3f4589c85870b60cbdc7b2c06, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:10,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289190848"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289190848"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289190848"}]},"ts":"1676289190848"} 2023-02-13 11:53:10,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 7e2e3da3f4589c85870b60cbdc7b2c06, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:11,017 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 7e2e3da3f4589c85870b60cbdc7b2c06, disabling compactions & flushes 2023-02-13 11:53:11,018 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:11,018 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:11,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. after waiting 0 ms 2023-02-13 11:53:11,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:11,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 7e2e3da3f4589c85870b60cbdc7b2c06 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:53:11,039 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/0370acf2557a4b67949e2eeaab441954 2023-02-13 11:53:11,067 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/940ad792297c478c921640fc9f1b198a 2023-02-13 11:53:11,073 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf1/0370acf2557a4b67949e2eeaab441954 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954 2023-02-13 11:53:11,080 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:11,082 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.tmp/cf2/940ad792297c478c921640fc9f1b198a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a 2023-02-13 11:53:11,091 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:11,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 73ms, sequenceid=82, compaction requested=true 2023-02-13 11:53:11,112 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:53:11,113 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:11,113 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. 2023-02-13 11:53:11,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 7e2e3da3f4589c85870b60cbdc7b2c06: Waiting for close lock at 1676289191018Running coprocessor pre-close hooks at 1676289191018Disabling compacts and flushes for region at 1676289191018Disabling writes for close at 1676289191019 (+1 ms)Obtaining lock to block concurrent updates at 1676289191019Preparing flush snapshotting stores in 7e2e3da3f4589c85870b60cbdc7b2c06 at 1676289191019Finished memstore snapshotting testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289191020 (+1 ms)Flushing stores of testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. at 1676289191021 (+1 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf1: creating writer at 1676289191022 (+1 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf1: appending metadata at 1676289191027 (+5 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf1: closing flushed file at 1676289191027Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf2: creating writer at 1676289191048 (+21 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf2: appending metadata at 1676289191053 (+5 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf2: closing flushed file at 1676289191053Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf1: reopening flushed file at 1676289191074 (+21 ms)Flushing 7e2e3da3f4589c85870b60cbdc7b2c06/cf2: reopening flushed file at 1676289191084 (+10 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 7e2e3da3f4589c85870b60cbdc7b2c06 in 73ms, sequenceid=82, compaction requested=true at 1676289191092 (+8 ms)Writing region close event to WAL at 1676289191108 (+16 ms)Running coprocessor post-close hooks at 1676289191113 (+5 ms)Closed at 1676289191113 2023-02-13 11:53:11,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,117 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=7e2e3da3f4589c85870b60cbdc7b2c06, regionState=CLOSED 2023-02-13 11:53:11,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289191116"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289191116"}]},"ts":"1676289191116"} 2023-02-13 11:53:11,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-02-13 11:53:11,122 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 7e2e3da3f4589c85870b60cbdc7b2c06, server=jenkins-hbase16.apache.org,43159,1676289185121 in 269 msec 2023-02-13 11:53:11,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-02-13 11:53:11,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7e2e3da3f4589c85870b60cbdc7b2c06, UNASSIGN in 276 msec 2023-02-13 11:53:11,144 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=7e2e3da3f4589c85870b60cbdc7b2c06, threads=14 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,145 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,146 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,146 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,146 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,147 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,146 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,147 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,147 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,147 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,172 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a, top=true 2023-02-13 11:53:11,174 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a, top=true 2023-02-13 11:53:11,179 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1, top=true 2023-02-13 11:53:11,181 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954, top=true 2023-02-13 11:53:11,182 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610, top=true 2023-02-13 11:53:11,182 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1, top=true 2023-02-13 11:53:11,217 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1 for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,218 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,232 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610 for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,232 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,233 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,233 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,235 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1 for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,235 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,236 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,237 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,241 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/.splits/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954 for child: bb81a7325747db4cafb6ad2501d634f0, parent: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,241 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,330 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,340 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,340 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,342 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,350 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,351 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,352 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,353 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1 for region: 7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:11,353 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 7e2e3da3f4589c85870b60cbdc7b2c06 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:11,401 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:11,403 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:11,405 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1676289191405"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1676289191405"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1676289191405"}]},"ts":"1676289191405"} 2023-02-13 11:53:11,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289191405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289191405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289191405"}]},"ts":"1676289191405"} 2023-02-13 11:53:11,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289191405"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289191405"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289191405"}]},"ts":"1676289191405"} 2023-02-13 11:53:11,446 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, ASSIGN}] 2023-02-13 11:53:11,448 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, ASSIGN 2023-02-13 11:53:11,449 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:11,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, ASSIGN 2023-02-13 11:53:11,450 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:11,599 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:53:11,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=bb81a7325747db4cafb6ad2501d634f0, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:11,603 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289191602"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289191602"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289191602"}]},"ts":"1676289191602"} 2023-02-13 11:53:11,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure bb81a7325747db4cafb6ad2501d634f0, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:11,612 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5272e69e5105d902d94749c8ce874cdd, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:11,612 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289191612"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289191612"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289191612"}]},"ts":"1676289191612"} 2023-02-13 11:53:11,614 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 5272e69e5105d902d94749c8ce874cdd, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:11,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bb81a7325747db4cafb6ad2501d634f0, NAME => 'testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.', STARTKEY => '26', ENDKEY => ''} 2023-02-13 11:53:11,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:11,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:11,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,765 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,767 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,768 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1 2023-02-13 11:53:11,768 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1 2023-02-13 11:53:11,769 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb81a7325747db4cafb6ad2501d634f0 columnFamilyName cf1 2023-02-13 11:53:11,799 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-top 2023-02-13 11:53:11,807 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-top 2023-02-13 11:53:11,814 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-top 2023-02-13 11:53:11,821 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-top 2023-02-13 11:53:11,828 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954 2023-02-13 11:53:11,834 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1 2023-02-13 11:53:11,840 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a 2023-02-13 11:53:11,841 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(310): Store=bb81a7325747db4cafb6ad2501d634f0/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:11,841 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,842 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2 2023-02-13 11:53:11,843 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2 2023-02-13 11:53:11,843 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bb81a7325747db4cafb6ad2501d634f0 columnFamilyName cf2 2023-02-13 11:53:11,861 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-top 2023-02-13 11:53:11,868 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-top 2023-02-13 11:53:11,874 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-top 2023-02-13 11:53:11,880 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-top 2023-02-13 11:53:11,886 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1 2023-02-13 11:53:11,892 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a 2023-02-13 11:53:11,898 DEBUG [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610 2023-02-13 11:53:11,898 INFO [StoreOpener-bb81a7325747db4cafb6ad2501d634f0-1] regionserver.HStore(310): Store=bb81a7325747db4cafb6ad2501d634f0/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:11,899 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,901 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,903 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:11,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:11,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened bb81a7325747db4cafb6ad2501d634f0; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70289267, jitterRate=0.04739169776439667}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:11,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for bb81a7325747db4cafb6ad2501d634f0: Running coprocessor pre-open hook at 1676289191766Writing region info on filesystem at 1676289191766Initializing all the Stores at 1676289191767 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289191767Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289191767Cleaning up temporary data from old regions at 1676289191902 (+135 ms)Cleaning up detritus from prior splits at 1676289191902Running coprocessor post-open hooks at 1676289191906 (+4 ms)Region opened successfully at 1676289191906 2023-02-13 11:53:11,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0., pid=17, masterSystemTime=1676289191760 2023-02-13 11:53:11,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:11,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:11,911 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:11,911 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:11,914 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): bb81a7325747db4cafb6ad2501d634f0/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:11,914 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): bb81a7325747db4cafb6ad2501d634f0/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:11,914 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bb81a7325747db4cafb6ad2501d634f0/cf2 in testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bb81a7325747db4cafb6ad2501d634f0/cf1 in testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:11,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:11,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5272e69e5105d902d94749c8ce874cdd, NAME => 'testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.', STARTKEY => '', ENDKEY => '26'} 2023-02-13 11:53:11,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=bb81a7325747db4cafb6ad2501d634f0, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:11,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:11,915 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/.tmp, totalSize=34.9 K 2023-02-13 11:53:11,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:11,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289191915"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289191915"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289191915"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289191915"}]},"ts":"1676289191915"} 2023-02-13 11:53:11,915 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/.tmp, totalSize=34.9 K 2023-02-13 11:53:11,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:11,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:11,917 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:11,919 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289188696 2023-02-13 11:53:11,919 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289188696 2023-02-13 11:53:11,920 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:11,920 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289189186 2023-02-13 11:53:11,920 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289189186 2023-02-13 11:53:11,921 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289189494 2023-02-13 11:53:11,923 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289189494 2023-02-13 11:53:11,923 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289189833 2023-02-13 11:53:11,924 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1 2023-02-13 11:53:11,924 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1 2023-02-13 11:53:11,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-02-13 11:53:11,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure bb81a7325747db4cafb6ad2501d634f0, server=jenkins-hbase16.apache.org,43159,1676289185121 in 313 msec 2023-02-13 11:53:11,924 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289189833 2023-02-13 11:53:11,924 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5272e69e5105d902d94749c8ce874cdd columnFamilyName cf1 2023-02-13 11:53:11,925 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289190186 2023-02-13 11:53:11,925 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289190504 2023-02-13 11:53:11,927 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289190186 2023-02-13 11:53:11,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, ASSIGN in 478 msec 2023-02-13 11:53:11,928 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289190808 2023-02-13 11:53:11,928 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289190504 2023-02-13 11:53:11,929 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289190808 2023-02-13 11:53:11,946 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-bottom 2023-02-13 11:53:11,964 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-bottom 2023-02-13 11:53:11,981 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-bottom 2023-02-13 11:53:11,988 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): bb81a7325747db4cafb6ad2501d634f0#cf2#compaction#14 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:11,995 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-bottom 2023-02-13 11:53:11,996 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(310): Store=5272e69e5105d902d94749c8ce874cdd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:11,996 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:11,997 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): bb81a7325747db4cafb6ad2501d634f0#cf1#compaction#15 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:11,998 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2 2023-02-13 11:53:11,998 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2 2023-02-13 11:53:11,999 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5272e69e5105d902d94749c8ce874cdd columnFamilyName cf2 2023-02-13 11:53:12,019 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-bottom 2023-02-13 11:53:12,020 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/.tmp/cf2/c7c7b58417324b9f8d6259a9485c0948 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/c7c7b58417324b9f8d6259a9485c0948 2023-02-13 11:53:12,024 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-bottom 2023-02-13 11:53:12,029 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-bottom 2023-02-13 11:53:12,034 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb81a7325747db4cafb6ad2501d634f0/cf2 of bb81a7325747db4cafb6ad2501d634f0 into c7c7b58417324b9f8d6259a9485c0948(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:12,034 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb81a7325747db4cafb6ad2501d634f0: 2023-02-13 11:53:12,034 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0., storeName=bb81a7325747db4cafb6ad2501d634f0/cf2, priority=9, startTime=1676289191910; duration=0sec 2023-02-13 11:53:12,035 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:12,036 DEBUG [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-bottom 2023-02-13 11:53:12,036 INFO [StoreOpener-5272e69e5105d902d94749c8ce874cdd-1] regionserver.HStore(310): Store=5272e69e5105d902d94749c8ce874cdd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:12,038 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:12,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,049 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/.tmp/cf1/1dc5f2331ca74482bea57b9a0d58db02 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1dc5f2331ca74482bea57b9a0d58db02 2023-02-13 11:53:12,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5272e69e5105d902d94749c8ce874cdd; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65466414, jitterRate=-0.024474412202835083}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:12,049 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5272e69e5105d902d94749c8ce874cdd: Running coprocessor pre-open hook at 1676289191917Writing region info on filesystem at 1676289191917Initializing all the Stores at 1676289191920 (+3 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289191920Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289191920Cleaning up temporary data from old regions at 1676289192042 (+122 ms)Cleaning up detritus from prior splits at 1676289192042Running coprocessor post-open hooks at 1676289192049 (+7 ms)Region opened successfully at 1676289192049 2023-02-13 11:53:12,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd., pid=18, masterSystemTime=1676289191760 2023-02-13 11:53:12,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:12,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:12,052 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 11:53:12,052 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,052 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): 5272e69e5105d902d94749c8ce874cdd/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:12,052 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5272e69e5105d902d94749c8ce874cdd/cf1 in testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,053 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/.tmp, totalSize=19.9 K 2023-02-13 11:53:12,054 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289188696 2023-02-13 11:53:12,054 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289189186 2023-02-13 11:53:12,055 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,055 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289189494 2023-02-13 11:53:12,056 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289189833 2023-02-13 11:53:12,056 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=5272e69e5105d902d94749c8ce874cdd, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:12,056 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289192056"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289192056"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289192056"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289192056"}]},"ts":"1676289192056"} 2023-02-13 11:53:12,061 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bb81a7325747db4cafb6ad2501d634f0/cf1 of bb81a7325747db4cafb6ad2501d634f0 into 1dc5f2331ca74482bea57b9a0d58db02(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:12,061 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bb81a7325747db4cafb6ad2501d634f0: 2023-02-13 11:53:12,061 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0., storeName=bb81a7325747db4cafb6ad2501d634f0/cf1, priority=9, startTime=1676289191909; duration=0sec 2023-02-13 11:53:12,061 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:12,061 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-02-13 11:53:12,062 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,062 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): 5272e69e5105d902d94749c8ce874cdd/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:12,062 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5272e69e5105d902d94749c8ce874cdd/cf2 in testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,062 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/.tmp, totalSize=19.9 K 2023-02-13 11:53:12,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-02-13 11:53:12,063 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289188696 2023-02-13 11:53:12,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 5272e69e5105d902d94749c8ce874cdd, server=jenkins-hbase16.apache.org,43159,1676289185121 in 445 msec 2023-02-13 11:53:12,064 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289189186 2023-02-13 11:53:12,065 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289189494 2023-02-13 11:53:12,066 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289189833 2023-02-13 11:53:12,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=15, resume processing ppid=12 2023-02-13 11:53:12,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, ASSIGN in 617 msec 2023-02-13 11:53:12,072 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 in 1.2370 sec 2023-02-13 11:53:12,073 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): 5272e69e5105d902d94749c8ce874cdd#cf1#compaction#16 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:12,080 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5272e69e5105d902d94749c8ce874cdd#cf2#compaction#17 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:12,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43159] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 188.40.62.62:45010 deadline: 1676289252090, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06. is not online on jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:12,092 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/.tmp/cf1/9405ee260333483f8280e634e6739cad as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/9405ee260333483f8280e634e6739cad 2023-02-13 11:53:12,096 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/.tmp/cf2/1769d2687ddd49808ad2cc0dda3658e4 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/1769d2687ddd49808ad2cc0dda3658e4 2023-02-13 11:53:12,098 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5272e69e5105d902d94749c8ce874cdd/cf1 of 5272e69e5105d902d94749c8ce874cdd into 9405ee260333483f8280e634e6739cad(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:12,098 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5272e69e5105d902d94749c8ce874cdd: 2023-02-13 11:53:12,099 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd., storeName=5272e69e5105d902d94749c8ce874cdd/cf1, priority=12, startTime=1676289192051; duration=0sec 2023-02-13 11:53:12,099 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:12,104 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 5272e69e5105d902d94749c8ce874cdd/cf2 of 5272e69e5105d902d94749c8ce874cdd into 1769d2687ddd49808ad2cc0dda3658e4(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:12,104 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5272e69e5105d902d94749c8ce874cdd: 2023-02-13 11:53:12,104 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd., storeName=5272e69e5105d902d94749c8ce874cdd/cf2, priority=12, startTime=1676289192051; duration=0sec 2023-02-13 11:53:12,104 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:12,237 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 11:53:12,252 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 11:53:12,255 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-02-13 11:53:12,285 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:12,295 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,311 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289192310"}]},"ts":"1676289192310"} 2023-02-13 11:53:12,313 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-02-13 11:53:12,317 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 11:53:12,350 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-02-13 11:53:12,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, UNASSIGN}] 2023-02-13 11:53:12,353 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, UNASSIGN 2023-02-13 11:53:12,354 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=bb81a7325747db4cafb6ad2501d634f0, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:12,354 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289192354"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289192354"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289192354"}]},"ts":"1676289192354"} 2023-02-13 11:53:12,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure bb81a7325747db4cafb6ad2501d634f0, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:12,360 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, UNASSIGN 2023-02-13 11:53:12,361 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5272e69e5105d902d94749c8ce874cdd, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:12,362 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289192361"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289192361"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289192361"}]},"ts":"1676289192361"} 2023-02-13 11:53:12,363 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 5272e69e5105d902d94749c8ce874cdd, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:12,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 11:53:12,488 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:12,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5272e69e5105d902d94749c8ce874cdd, disabling compactions & flushes 2023-02-13 11:53:12,510 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. after waiting 0 ms 2023-02-13 11:53:12,510 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,515 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-bottom] to archive 2023-02-13 11:53:12,516 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:12,522 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,523 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,525 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,527 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,554 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-bottom] to archive 2023-02-13 11:53:12,555 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:12,558 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,560 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,562 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,566 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:53:12,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:12,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd. 2023-02-13 11:53:12,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5272e69e5105d902d94749c8ce874cdd: Waiting for close lock at 1676289192510Running coprocessor pre-close hooks at 1676289192510Disabling compacts and flushes for region at 1676289192510Disabling writes for close at 1676289192510Writing region close event to WAL at 1676289192568 (+58 ms)Running coprocessor post-close hooks at 1676289192575 (+7 ms)Closed at 1676289192576 (+1 ms) 2023-02-13 11:53:12,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:12,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing bb81a7325747db4cafb6ad2501d634f0, disabling compactions & flushes 2023-02-13 11:53:12,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:12,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:12,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. after waiting 0 ms 2023-02-13 11:53:12,578 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:12,579 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=5272e69e5105d902d94749c8ce874cdd, regionState=CLOSED 2023-02-13 11:53:12,579 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289192579"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289192579"}]},"ts":"1676289192579"} 2023-02-13 11:53:12,580 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954] to archive 2023-02-13 11:53:12,582 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:12,585 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/e04618c2739d4282852e3645e745000d.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-02-13 11:53:12,585 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 5272e69e5105d902d94749c8ce874cdd, server=jenkins-hbase16.apache.org,43159,1676289185121 in 218 msec 2023-02-13 11:53:12,587 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/810018f574264e0093ef351705f6d615.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=5272e69e5105d902d94749c8ce874cdd, UNASSIGN in 233 msec 2023-02-13 11:53:12,589 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/083efa3a87854ee8b4c5330d9da542a1.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,590 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1506dbc99dba455e8a16e5d157481631.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,592 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-437ed8d6d87a46b980251f1551e72bb1 2023-02-13 11:53:12,593 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-5473d4d9e16f4a2e891dd2ef6366401a 2023-02-13 11:53:12,594 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-0370acf2557a4b67949e2eeaab441954 2023-02-13 11:53:12,595 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-02-13 11:53:12,596 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-02-13 11:53:12,596 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a] to archive 2023-02-13 11:53:12,597 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:12,599 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/0fc1027ce690420fae581963c26cbeba.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,600 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/55d1504fa10340b08761b300867a61ca.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,602 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/841e7d25e69e432688c388a5c83d9a7f.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,603 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/e3ee0f0b47024188b03604aebb88d01a.7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,604 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-c3cecfde6e8449889ef537b61bc6b610 2023-02-13 11:53:12,606 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-2fba25ef607c4885b5ba3bd1305fd0d1 2023-02-13 11:53:12,607 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/testSplitTableRegionUnevenDaughter=7e2e3da3f4589c85870b60cbdc7b2c06-940ad792297c478c921640fc9f1b198a 2023-02-13 11:53:12,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:53:12,612 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:12,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0. 2023-02-13 11:53:12,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for bb81a7325747db4cafb6ad2501d634f0: Waiting for close lock at 1676289192578Running coprocessor pre-close hooks at 1676289192578Disabling compacts and flushes for region at 1676289192578Disabling writes for close at 1676289192578Writing region close event to WAL at 1676289192608 (+30 ms)Running coprocessor post-close hooks at 1676289192612 (+4 ms)Closed at 1676289192614 (+2 ms) 2023-02-13 11:53:12,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:12,616 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=bb81a7325747db4cafb6ad2501d634f0, regionState=CLOSED 2023-02-13 11:53:12,617 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1676289192616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289192616"}]},"ts":"1676289192616"} 2023-02-13 11:53:12,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-02-13 11:53:12,622 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure bb81a7325747db4cafb6ad2501d634f0, server=jenkins-hbase16.apache.org,43159,1676289185121 in 263 msec 2023-02-13 11:53:12,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 11:53:12,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-02-13 11:53:12,624 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=bb81a7325747db4cafb6ad2501d634f0, UNASSIGN in 270 msec 2023-02-13 11:53:12,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289192625"}]},"ts":"1676289192625"} 2023-02-13 11:53:12,627 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-02-13 11:53:12,635 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-02-13 11:53:12,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 336 msec 2023-02-13 11:53:12,925 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-02-13 11:53:12,927 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-02-13 11:53:12,935 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,941 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,943 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,945 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:12,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 11:53:12,952 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,952 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:12,952 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,955 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/recovered.edits] 2023-02-13 11:53:12,955 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/recovered.edits] 2023-02-13 11:53:12,955 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/recovered.edits] 2023-02-13 11:53:12,959 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1dc5f2331ca74482bea57b9a0d58db02 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf1/1dc5f2331ca74482bea57b9a0d58db02 2023-02-13 11:53:12,959 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/0370acf2557a4b67949e2eeaab441954 2023-02-13 11:53:12,960 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/9405ee260333483f8280e634e6739cad to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf1/9405ee260333483f8280e634e6739cad 2023-02-13 11:53:12,961 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/083efa3a87854ee8b4c5330d9da542a1 2023-02-13 11:53:12,963 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/c7c7b58417324b9f8d6259a9485c0948 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/cf2/c7c7b58417324b9f8d6259a9485c0948 2023-02-13 11:53:12,964 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/1506dbc99dba455e8a16e5d157481631 2023-02-13 11:53:12,964 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/1769d2687ddd49808ad2cc0dda3658e4 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/cf2/1769d2687ddd49808ad2cc0dda3658e4 2023-02-13 11:53:12,966 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/437ed8d6d87a46b980251f1551e72bb1 2023-02-13 11:53:12,967 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0/recovered.edits/90.seqid 2023-02-13 11:53:12,968 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd/recovered.edits/90.seqid 2023-02-13 11:53:12,968 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:12,968 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/5473d4d9e16f4a2e891dd2ef6366401a 2023-02-13 11:53:12,968 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/5272e69e5105d902d94749c8ce874cdd 2023-02-13 11:53:12,969 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/810018f574264e0093ef351705f6d615 2023-02-13 11:53:12,971 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf1/e04618c2739d4282852e3645e745000d 2023-02-13 11:53:12,974 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/0fc1027ce690420fae581963c26cbeba 2023-02-13 11:53:12,975 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/2fba25ef607c4885b5ba3bd1305fd0d1 2023-02-13 11:53:12,977 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/55d1504fa10340b08761b300867a61ca 2023-02-13 11:53:12,979 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/841e7d25e69e432688c388a5c83d9a7f 2023-02-13 11:53:12,981 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/940ad792297c478c921640fc9f1b198a 2023-02-13 11:53:12,982 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/c3cecfde6e8449889ef537b61bc6b610 2023-02-13 11:53:12,984 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/cf2/e3ee0f0b47024188b03604aebb88d01a 2023-02-13 11:53:12,987 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06/recovered.edits/85.seqid 2023-02-13 11:53:12,987 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionUnevenDaughter/7e2e3da3f4589c85870b60cbdc7b2c06 2023-02-13 11:53:12,987 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-02-13 11:53:12,990 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:13,007 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-02-13 11:53:13,043 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-02-13 11:53:13,044 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:13,044 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-02-13 11:53:13,044 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289193044"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:13,044 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289193044"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:13,044 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289193044"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:13,047 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:53:13,047 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 7e2e3da3f4589c85870b60cbdc7b2c06, NAME => 'testSplitTableRegionUnevenDaughter,,1676289188184.7e2e3da3f4589c85870b60cbdc7b2c06.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 5272e69e5105d902d94749c8ce874cdd, NAME => 'testSplitTableRegionUnevenDaughter,,1676289190831.5272e69e5105d902d94749c8ce874cdd.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => bb81a7325747db4cafb6ad2501d634f0, NAME => 'testSplitTableRegionUnevenDaughter,26,1676289190831.bb81a7325747db4cafb6ad2501d634f0.', STARTKEY => '26', ENDKEY => ''}] 2023-02-13 11:53:13,047 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-02-13 11:53:13,047 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289193047"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:13,048 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 11:53:13,049 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-02-13 11:53:13,084 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:13,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 149 msec 2023-02-13 11:53:13,249 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-02-13 11:53:13,250 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-02-13 11:53:13,265 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=392 (was 360) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:56540 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52588 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase16:43159-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52548 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:56620 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:55446 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52556 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:54466 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:48134 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:60052 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:54484 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52612 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52586 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:48084 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52572 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52372 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2023632964_15 at /127.0.0.1:52314 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=851 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=309 (was 319), ProcessCount=167 (was 169), AvailableMemoryMB=6288 (was 6100) - AvailableMemoryMB LEAK? - 2023-02-13 11:53:13,279 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=392, OpenFileDescriptor=851, MaxFileDescriptor=60000, SystemLoadAverage=309, ProcessCount=169, AvailableMemoryMB=6283 2023-02-13 11:53:13,282 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:13,315 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:13,316 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:13,317 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:13,320 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:13,321 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:13,330 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,331 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de empty. 2023-02-13 11:53:13,331 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,331 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 11:53:13,361 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:13,363 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7671): creating {ENCODED => 00e2700a0258d2e58039799b308648de, NAME => 'testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1603): Closing 00e2700a0258d2e58039799b308648de, disabling compactions & flushes 2023-02-13 11:53:13,379 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. after waiting 0 ms 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,379 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,379 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1557): Region close journal for 00e2700a0258d2e58039799b308648de: Waiting for close lock at 1676289193379Disabling compacts and flushes for region at 1676289193379Disabling writes for close at 1676289193379Writing region close event to WAL at 1676289193379Closed at 1676289193379 2023-02-13 11:53:13,384 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:13,385 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289193385"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289193385"}]},"ts":"1676289193385"} 2023-02-13 11:53:13,387 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:13,388 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:13,388 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289193388"}]},"ts":"1676289193388"} 2023-02-13 11:53:13,390 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-02-13 11:53:13,411 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:13,412 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:13,412 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:13,412 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:13,412 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:13,412 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, ASSIGN}] 2023-02-13 11:53:13,415 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, ASSIGN 2023-02-13 11:53:13,416 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:53:13,566 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:13,568 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,568 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289193567"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289193567"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289193567"}]},"ts":"1676289193567"} 2023-02-13 11:53:13,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:13,722 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,723 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 11:53:13,726 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44486, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 11:53:13,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 00e2700a0258d2e58039799b308648de, NAME => 'testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:13,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:13,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:13,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,736 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,738 DEBUG [StoreOpener-00e2700a0258d2e58039799b308648de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1 2023-02-13 11:53:13,738 DEBUG [StoreOpener-00e2700a0258d2e58039799b308648de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1 2023-02-13 11:53:13,738 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00e2700a0258d2e58039799b308648de columnFamilyName cf1 2023-02-13 11:53:13,739 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] regionserver.HStore(310): Store=00e2700a0258d2e58039799b308648de/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:13,739 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,741 DEBUG [StoreOpener-00e2700a0258d2e58039799b308648de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2 2023-02-13 11:53:13,741 DEBUG [StoreOpener-00e2700a0258d2e58039799b308648de-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2 2023-02-13 11:53:13,743 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 00e2700a0258d2e58039799b308648de columnFamilyName cf2 2023-02-13 11:53:13,744 INFO [StoreOpener-00e2700a0258d2e58039799b308648de-1] regionserver.HStore(310): Store=00e2700a0258d2e58039799b308648de/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:13,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,745 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:13,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:13,755 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:13,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 00e2700a0258d2e58039799b308648de; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67021201, jitterRate=-0.001306280493736267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:13,756 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 00e2700a0258d2e58039799b308648de: Running coprocessor pre-open hook at 1676289193732Writing region info on filesystem at 1676289193732Initializing all the Stores at 1676289193733 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289193733Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289193736 (+3 ms)Cleaning up temporary data from old regions at 1676289193746 (+10 ms)Cleaning up detritus from prior splits at 1676289193747 (+1 ms)Running coprocessor post-open hooks at 1676289193756 (+9 ms)Region opened successfully at 1676289193756 2023-02-13 11:53:13,757 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de., pid=27, masterSystemTime=1676289193722 2023-02-13 11:53:13,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,763 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,763 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289193763"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289193763"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289193763"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289193763"}]},"ts":"1676289193763"} 2023-02-13 11:53:13,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-02-13 11:53:13,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 in 195 msec 2023-02-13 11:53:13,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-02-13 11:53:13,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, ASSIGN in 356 msec 2023-02-13 11:53:13,772 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:13,772 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289193772"}]},"ts":"1676289193772"} 2023-02-13 11:53:13,774 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-02-13 11:53:13,784 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:13,786 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 468 msec 2023-02-13 11:53:13,821 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 11:53:13,825 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:44498, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 11:53:13,834 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:13,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:13,837 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:13,837 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:13,837 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:13,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:13,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:13,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,874 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:13,874 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:13,874 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:13,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:13,874 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:13,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:13,875 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,875 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:13,876 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:13,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:13,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:13,879 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:13,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:13,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:13,879 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:13,879 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:13,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:13,880 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:13,880 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:13,881 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:13,881 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:13,882 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:13,882 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:13,882 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:13,882 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:13,884 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:13,884 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:13,884 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,884 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:13,885 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:53:13,891 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,899 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,899 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,900 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:13,900 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,900 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:13,916 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/676353ee928f426cb1a9ee45791c521a 2023-02-13 11:53:13,946 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/13baf99673e045eebc4a13bc9f309c8c 2023-02-13 11:53:13,952 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/676353ee928f426cb1a9ee45791c521a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a 2023-02-13 11:53:13,958 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:13,958 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/13baf99673e045eebc4a13bc9f309c8c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c 2023-02-13 11:53:13,963 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:13,964 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 79ms, sequenceid=6, compaction requested=false 2023-02-13 11:53:13,964 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-02-13 11:53:13,965 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:13,965 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:13,965 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:13,965 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:13,965 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,965 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:13,965 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:13,974 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,974 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,974 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:13,975 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,975 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:13,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:13,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:13,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:13,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:13,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:13,977 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:13,977 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4c53e3e1[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:13,977 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,985 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,985 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,985 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:13,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:13,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:13,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:13,986 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:13,994 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:13,994 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:13,994 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,002 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,002 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,002 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,002 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,002 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,003 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:14,008 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:14,008 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:14,008 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:14,008 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:14,008 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:14,008 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:14,016 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,016 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,016 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,016 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,016 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,017 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,017 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,019 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,019 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,020 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,029 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:14,029 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:14,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,029 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,029 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:14,029 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitWithoutPONR because node does not exist (not an error) 2023-02-13 11:53:14,029 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,030 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,029 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:14,030 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,030 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,030 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,030 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,130 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:14,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:14,152 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:14,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:14,156 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:14,156 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:14,156 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:14,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:14,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,166 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,167 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,167 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,169 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,171 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,171 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,171 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:14,172 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,172 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:14,172 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,172 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,172 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,172 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:14,173 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:14,183 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,188 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/352a24241fb04817b9724f9a0766b80d 2023-02-13 11:53:14,191 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,191 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,191 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,191 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,191 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,203 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 11:53:14,203 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-02-13 11:53:14,203 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:14,203 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-02-13 11:53:14,220 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/67664b1791514f50894e6ebbc320a3b0 2023-02-13 11:53:14,227 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/352a24241fb04817b9724f9a0766b80d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d 2023-02-13 11:53:14,233 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:14,234 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/67664b1791514f50894e6ebbc320a3b0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 2023-02-13 11:53:14,240 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:14,241 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 68ms, sequenceid=19, compaction requested=false 2023-02-13 11:53:14,241 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:14,241 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:14,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:14,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,242 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,252 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,252 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,253 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,253 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:14,255 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@25aef210[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:14,255 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:14,255 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,261 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:14,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:14,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:14,261 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:14,300 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,300 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,300 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,310 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,311 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,311 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,311 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,312 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:14,317 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:14,317 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:14,317 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:14,317 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:14,317 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:14,317 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:14,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,327 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,327 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,328 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,328 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,328 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,330 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,333 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,333 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,333 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:14,344 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:14,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,344 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:14,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,345 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,346 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,346 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,346 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,445 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:14,447 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:14,475 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:14,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:14,478 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:14,478 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:14,478 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,479 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:14,479 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:14,479 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,479 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,486 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,486 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:14,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,489 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,489 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,489 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,489 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,489 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,489 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,489 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,495 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,495 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,495 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,495 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:14,496 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,496 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:14,496 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,497 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,497 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,497 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,497 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:14,502 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,511 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,511 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,514 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/376a34289efe48bd8b6cf6a48becdcd7 2023-02-13 11:53:14,538 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/f7e1d14bb6114288aece80770e5cbee8 2023-02-13 11:53:14,543 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/376a34289efe48bd8b6cf6a48becdcd7 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 2023-02-13 11:53:14,551 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:14,552 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/f7e1d14bb6114288aece80770e5cbee8 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 2023-02-13 11:53:14,565 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:14,566 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 70ms, sequenceid=32, compaction requested=true 2023-02-13 11:53:14,567 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:14,567 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,567 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:14,567 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:14,567 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,567 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,567 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,577 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,577 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,578 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,578 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:14,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31c42f41[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:14,579 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:14,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:14,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,586 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,586 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,587 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,594 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,602 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,602 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,602 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,603 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:14,606 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:14,606 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:14,606 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:14,606 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:14,606 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:14,607 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,617 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,617 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,618 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,618 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,620 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,621 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,621 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:14,633 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:14,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:14,633 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,633 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,734 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:14,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:14,764 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:14,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:14,768 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:14,768 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:14,768 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,769 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:14,769 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:14,769 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,769 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,793 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,793 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,793 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,793 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,793 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,794 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:14,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:14,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,795 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,795 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,795 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:14,796 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,795 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,796 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,796 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:14,797 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,797 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,797 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:14,797 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,799 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:14,799 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,799 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:14,799 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:14,799 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,799 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:14,799 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:14,810 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,814 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/c70362f983a14f8ca6bc19efc048546e 2023-02-13 11:53:14,819 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,819 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,819 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,819 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,819 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,827 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/c902e027cb294e428fc0ce1c657c13b3 2023-02-13 11:53:14,836 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/c70362f983a14f8ca6bc19efc048546e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e 2023-02-13 11:53:14,845 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:14,846 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/c902e027cb294e428fc0ce1c657c13b3 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 2023-02-13 11:53:14,851 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:14,859 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 59ms, sequenceid=45, compaction requested=true 2023-02-13 11:53:14,859 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:14,859 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:14,859 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:14,859 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:14,859 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,859 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:14,859 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:14,869 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,869 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,869 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,870 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:14,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:14,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@296c659a[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:14,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:14,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,878 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,878 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:14,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:14,878 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:14,885 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,885 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,886 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,886 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,886 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,886 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:14,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,895 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:14,898 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:14,898 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:14,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:14,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:14,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:14,898 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,908 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:14,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,909 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,909 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:14,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,910 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:14,910 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,911 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,912 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,912 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:14,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:14,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:14,919 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:14,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,919 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:14,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:14,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:14,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:14,920 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:14,920 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:14,921 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,020 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:15,021 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:15,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:15,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:15,045 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:15,045 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:15,045 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,045 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:15,045 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:15,046 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,046 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,066 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,066 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,067 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,067 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,067 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,068 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,068 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,068 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,068 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,069 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,069 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,069 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,069 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,069 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,069 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:15,069 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:15,069 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:15,069 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:15,069 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:15,079 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/8ee354f1470e40599fb318ba0a2205e0 2023-02-13 11:53:15,094 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 2023-02-13 11:53:15,100 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/8ee354f1470e40599fb318ba0a2205e0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 2023-02-13 11:53:15,105 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:15,106 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 2023-02-13 11:53:15,112 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:15,113 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 44ms, sequenceid=58, compaction requested=true 2023-02-13 11:53:15,113 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:15,113 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:15,113 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:15,113 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:15,113 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,113 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,113 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,116 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,133 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,133 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,133 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,134 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,134 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,134 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,134 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:15,137 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7af3c7bd[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:15,137 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:15,138 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,167 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,167 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:15,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,167 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:15,208 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,208 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,209 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,208 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,219 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,219 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,219 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,227 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,227 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,227 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:15,230 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:15,230 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:15,230 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:15,230 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:15,231 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:15,231 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:15,262 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,262 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,262 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,262 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,263 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,264 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,267 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,267 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,267 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,283 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:15,283 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:15,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:15,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:15,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,384 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:15,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:15,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:15,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:15,413 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-02-13 11:53:15,413 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-02-13 11:53:15,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-02-13 11:53:15,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-02-13 11:53:15,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,427 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,427 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,428 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,428 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,429 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-02-13 11:53:15,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,430 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-02-13 11:53:15,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,430 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,430 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-02-13 11:53:15,431 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:15,431 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:15,431 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:15,431 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. started... 2023-02-13 11:53:15,431 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:15,441 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,441 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,442 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,442 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,442 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,442 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,443 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/50de1a674b394cff96414e041962053f 2023-02-13 11:53:15,461 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/98bfdba0aa8e40e29830fa62b2318f21 2023-02-13 11:53:15,466 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/50de1a674b394cff96414e041962053f as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f 2023-02-13 11:53:15,472 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:15,473 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/98bfdba0aa8e40e29830fa62b2318f21 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 2023-02-13 11:53:15,479 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:15,480 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 49ms, sequenceid=71, compaction requested=true 2023-02-13 11:53:15,480 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 00e2700a0258d2e58039799b308648de: 2023-02-13 11:53:15,480 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:15,480 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:15,480 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:15,480 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,481 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-02-13 11:53:15,481 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-02-13 11:53:15,491 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,492 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,492 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:15,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-02-13 11:53:15,494 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@289f9481[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:15,494 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-02-13 11:53:15,494 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,503 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:15,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:15,503 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:15,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,511 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,519 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-02-13 11:53:15,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,519 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-02-13 11:53:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:53:15,523 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-02-13 11:53:15,523 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-02-13 11:53:15,523 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:15,523 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:15,523 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-02-13 11:53:15,523 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,533 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:15,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,534 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:15,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,536 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:15,537 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-02-13 11:53:15,537 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,537 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,538 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,538 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,544 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,544 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:15,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,544 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-02-13 11:53:15,545 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:15,544 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:15,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:15,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,545 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:15,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,545 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:15,545 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:15,545 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-02-13 11:53:15,546 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-02-13 11:53:15,645 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:15,647 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:15,910 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 11:53:15,911 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:15,913 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:15,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:15,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:15,914 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:15,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:15,915 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:15,915 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:15,915 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:16,163 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,165 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:16,165 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:16,165 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:16,168 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-02-13 11:53:16,169 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:16,169 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:16,169 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:16,170 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:16,188 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 18 msec 2023-02-13 11:53:16,214 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 11:53:16,216 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:16,216 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:16,217 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:16,219 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:16,220 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,221 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:16,222 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,222 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,223 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:16,224 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,224 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:16,224 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,224 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,224 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,225 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 36 msec 2023-02-13 11:53:16,227 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:16,228 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:16,231 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289226231 2023-02-13 11:53:16,231 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:16,233 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,238 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:16,239 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:16,239 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:16,239 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289256239 2023-02-13 11:53:16,239 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289316239 2023-02-13 11:53:16,239 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 7 msec 2023-02-13 11:53:16,240 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:16,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:16,243 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:16,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:16,490 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,490 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:16,490 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:16,490 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:16,491 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:16,491 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:16,491 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:16,491 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:16,492 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:16,493 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:16,494 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 11:53:16,495 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:16,495 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:16,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:16,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:16,496 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,497 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:16,497 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,497 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,497 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:16,498 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,498 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:16,498 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,498 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,499 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,499 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:53:16,500 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:16,500 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:16,504 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289226504 2023-02-13 11:53:16,506 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,506 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:16,509 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:16,510 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:16,510 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:16,510 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289256510 2023-02-13 11:53:16,510 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289316510 2023-02-13 11:53:16,510 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:16,510 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:16,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN}] 2023-02-13 11:53:16,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:16,514 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:16,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:16,760 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,760 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:16,760 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:16,761 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:16,761 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:16,762 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:16,762 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:16,762 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:16,762 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:16,762 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:16,764 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 11:53:16,764 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:16,765 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:16,765 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:16,765 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:16,766 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,766 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:16,767 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,767 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:16,767 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:16,768 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,768 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:16,768 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,768 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,768 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,768 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 11:53:16,769 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:16,769 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:16,772 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289226772 2023-02-13 11:53:16,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:16,773 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:16,776 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:16,777 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:16,777 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:16,777 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289256777 2023-02-13 11:53:16,777 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289316777 2023-02-13 11:53:16,777 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:16,777 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN}] 2023-02-13 11:53:16,778 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:16,780 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:16,781 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:16,781 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289196781"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289196781"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289196781"}]},"ts":"1676289196781"} 2023-02-13 11:53:16,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:16,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:16,783 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:16,783 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:16,783 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:17,027 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,028 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:17,028 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:17,028 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:17,029 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:17,029 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:17,029 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:17,029 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:17,029 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:17,030 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:17,031 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:17,031 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:17,032 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:17,032 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:17,032 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:17,032 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:17,033 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,033 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:17,033 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,034 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,034 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:17,035 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,035 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,035 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,035 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,035 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,035 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,035 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,035 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,035 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 11:53:17,036 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:17,037 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:17,037 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de to restore RIT 2023-02-13 11:53:17,037 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289227037 2023-02-13 11:53:17,038 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:17,038 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,041 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:17,041 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSING, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:17,041 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:17,041 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289257041 2023-02-13 11:53:17,041 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289317041 2023-02-13 11:53:17,041 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:17,042 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,042 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289197042"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289197042"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289197042"}]},"ts":"1676289197042"} 2023-02-13 11:53:17,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:17,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:17,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 00e2700a0258d2e58039799b308648de, disabling compactions & flushes 2023-02-13 11:53:17,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:17,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:17,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. after waiting 0 ms 2023-02-13 11:53:17,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:17,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 00e2700a0258d2e58039799b308648de 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:53:17,209 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 2023-02-13 11:53:17,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 2023-02-13 11:53:17,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 2023-02-13 11:53:17,242 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:17,243 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/.tmp/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 2023-02-13 11:53:17,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:17,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 55ms, sequenceid=82, compaction requested=true 2023-02-13 11:53:17,269 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:53:17,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:17,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. 2023-02-13 11:53:17,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 00e2700a0258d2e58039799b308648de: Waiting for close lock at 1676289197198Running coprocessor pre-close hooks at 1676289197198Disabling compacts and flushes for region at 1676289197198Disabling writes for close at 1676289197198Obtaining lock to block concurrent updates at 1676289197198Preparing flush snapshotting stores in 00e2700a0258d2e58039799b308648de at 1676289197198Finished memstore snapshotting testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289197198Flushing stores of testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. at 1676289197199 (+1 ms)Flushing 00e2700a0258d2e58039799b308648de/cf1: creating writer at 1676289197199Flushing 00e2700a0258d2e58039799b308648de/cf1: appending metadata at 1676289197202 (+3 ms)Flushing 00e2700a0258d2e58039799b308648de/cf1: closing flushed file at 1676289197202Flushing 00e2700a0258d2e58039799b308648de/cf2: creating writer at 1676289197215 (+13 ms)Flushing 00e2700a0258d2e58039799b308648de/cf2: appending metadata at 1676289197218 (+3 ms)Flushing 00e2700a0258d2e58039799b308648de/cf2: closing flushed file at 1676289197218Flushing 00e2700a0258d2e58039799b308648de/cf1: reopening flushed file at 1676289197237 (+19 ms)Flushing 00e2700a0258d2e58039799b308648de/cf2: reopening flushed file at 1676289197244 (+7 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 00e2700a0258d2e58039799b308648de in 55ms, sequenceid=82, compaction requested=true at 1676289197253 (+9 ms)Writing region close event to WAL at 1676289197266 (+13 ms)Running coprocessor post-close hooks at 1676289197270 (+4 ms)Closed at 1676289197270 2023-02-13 11:53:17,273 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,273 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=CLOSED 2023-02-13 11:53:17,274 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289197273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289197273"}]},"ts":"1676289197273"} 2023-02-13 11:53:17,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:17,275 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:17,275 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:17,292 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:17,292 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:17,293 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:17,293 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:17,295 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:17,295 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 11:53:17,295 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 11:53:17,295 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:17,296 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:17,296 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:17,296 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:17,297 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,298 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:17,298 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,298 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,299 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,300 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,300 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,300 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,300 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,300 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,301 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:17,301 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:17,302 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:17,302 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de to restore RIT 2023-02-13 11:53:17,302 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289227302 2023-02-13 11:53:17,303 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:17,303 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,306 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:17,306 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:17,307 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:17,307 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289257307 2023-02-13 11:53:17,307 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289317307 2023-02-13 11:53:17,307 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:17,307 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=00e2700a0258d2e58039799b308648de, regionState=CLOSED, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,307 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289197307"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289197307"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289197307"}]},"ts":"1676289197307"} 2023-02-13 11:53:17,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:17,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-02-13 11:53:17,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 in 265 msec 2023-02-13 11:53:17,312 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:17,312 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,312 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:17,312 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:17,418 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:17,557 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,557 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:17,558 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:17,558 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:17,558 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:17,558 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:17,558 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:17,558 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:17,559 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:17,559 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:17,562 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:17,563 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 11:53:17,563 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:17,563 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:17,565 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:17,565 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:17,565 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:17,566 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:17,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,568 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:17,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,569 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,570 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:17,570 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,570 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,570 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,570 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:17,570 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:17,570 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 11:53:17,571 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:17,571 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:17,572 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de to restore RIT 2023-02-13 11:53:17,572 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289227572 2023-02-13 11:53:17,573 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:17,573 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,575 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:17,575 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:17,575 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:17,575 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289257575 2023-02-13 11:53:17,575 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289317575 2023-02-13 11:53:17,575 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:17,576 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,577 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:17,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-02-13 11:53:17,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN in 800 msec 2023-02-13 11:53:17,583 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:17,584 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:17,584 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:17,826 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,826 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:17,826 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:17,826 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:17,828 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:17,828 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:17,828 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:17,828 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:17,828 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:17,829 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:17,831 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:17,831 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:17,831 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:17,831 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:17,831 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:17,832 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:17,832 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:17,832 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,833 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:17,833 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,833 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:17,834 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:17,834 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,835 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:17,835 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:17,835 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:17,835 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:17,835 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:17,835 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 11:53:17,836 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:17,837 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:17,842 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289227842 2023-02-13 11:53:17,843 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,843 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:17,846 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:17,846 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:17,846 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:17,846 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289257846 2023-02-13 11:53:17,846 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289317846 2023-02-13 11:53:17,846 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:17,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,847 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:17,853 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:17,863 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=00e2700a0258d2e58039799b308648de, threads=14 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,865 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,866 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,866 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,868 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,868 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,869 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,872 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:17,872 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,089 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,100 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,104 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,120 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,128 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,150 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,152 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,152 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,156 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,159 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,159 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,162 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,162 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,164 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,164 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 00e2700a0258d2e58039799b308648de Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:18,200 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:18,200 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:18,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:18,347 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,347 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:18,347 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:18,347 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:18,347 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:18,348 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:18,348 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:18,348 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:18,348 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:18,350 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:18,353 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 11:53:18,353 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:18,354 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:18,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:18,355 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:18,356 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:18,357 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:18,358 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:18,359 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:18,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:18,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:18,360 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:18,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:18,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:18,361 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:18,361 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,361 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,362 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,362 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:18,362 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:18,362 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 11:53:18,363 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:18,363 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:18,363 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289228363 2023-02-13 11:53:18,365 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,365 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:18,367 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:18,367 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:18,367 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:18,367 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289258367 2023-02-13 11:53:18,367 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289318367 2023-02-13 11:53:18,367 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:18,378 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=00e2700a0258d2e58039799b308648de, threads=14 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,380 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,384 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,384 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,384 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,384 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,390 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,390 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,390 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,391 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,510 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,525 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,535 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,535 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,553 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,556 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,557 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,561 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,561 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,563 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,564 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,565 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,567 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,908 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 for region: 00e2700a0258d2e58039799b308648de 2023-02-13 11:53:18,908 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 00e2700a0258d2e58039799b308648de Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:18,950 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:18,956 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:18,958 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:18,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:18,958 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:18,958 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:18,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:19,118 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:19,118 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:19,119 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:19,120 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 11:53:19,120 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:19,121 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:19,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:19,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:19,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:19,121 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:19,122 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,122 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:19,122 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,122 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,123 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:19,123 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,123 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:19,123 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:19,123 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:19,123 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,123 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,124 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:19,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:19,124 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 11:53:19,125 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:19,125 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:19,125 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289229125 2023-02-13 11:53:19,126 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:19,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,129 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:19,129 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:19,129 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:19,129 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289259129 2023-02-13 11:53:19,129 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289319129 2023-02-13 11:53:19,129 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:19,132 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:19,134 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:19,135 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:19,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:19,380 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:19,380 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:19,380 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:19,381 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:19,382 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 11:53:19,382 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:19,382 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:19,382 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:19,383 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:19,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:19,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN 2023-02-13 11:53:19,384 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:19,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:19,385 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,385 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,385 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=00e2700a0258d2e58039799b308648de, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:19,385 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 00e2700a0258d2e58039799b308648de, server=jenkins-hbase16.apache.org,44985,1676289185170 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:19,385 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 11:53:19,386 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:19,386 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:19,387 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289229387 2023-02-13 11:53:19,388 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:19,388 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,390 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:19,391 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=00e2700a0258d2e58039799b308648de, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44985,1676289185170, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170, openSeqNum=2 2023-02-13 11:53:19,391 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:19,391 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289259391 2023-02-13 11:53:19,391 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289319391 2023-02-13 11:53:19,391 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:19,394 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289199394"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676289199394"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676289199394"}]},"ts":"1676289199394"} 2023-02-13 11:53:19,394 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289199394"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289199394"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289199394"}]},"ts":"1676289199394"} 2023-02-13 11:53:19,395 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289199394"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289199394"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289199394"}]},"ts":"1676289199394"} 2023-02-13 11:53:19,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, ASSIGN}] 2023-02-13 11:53:19,402 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, ASSIGN 2023-02-13 11:53:19,403 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:53:19,404 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, ASSIGN 2023-02-13 11:53:19,405 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:53:19,553 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:53:19,555 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=873f0d131aa7d6fc989e0bdac352dc63, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:19,555 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289199555"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289199555"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289199555"}]},"ts":"1676289199555"} 2023-02-13 11:53:19,557 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=32, state=RUNNABLE; OpenRegionProcedure 873f0d131aa7d6fc989e0bdac352dc63, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:19,560 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=553d82e2c9cc5115f61d7f068d12f829, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:19,560 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289199560"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289199560"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289199560"}]},"ts":"1676289199560"} 2023-02-13 11:53:19,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=31, state=RUNNABLE; OpenRegionProcedure 553d82e2c9cc5115f61d7f068d12f829, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:19,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,713 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 873f0d131aa7d6fc989e0bdac352dc63, NAME => 'testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 11:53:19,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:19,714 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,715 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,717 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,718 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1 2023-02-13 11:53:19,718 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1 2023-02-13 11:53:19,719 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 873f0d131aa7d6fc989e0bdac352dc63 columnFamilyName cf1 2023-02-13 11:53:19,741 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-top 2023-02-13 11:53:19,747 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-top 2023-02-13 11:53:19,753 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-top 2023-02-13 11:53:19,759 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-top 2023-02-13 11:53:19,764 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-top 2023-02-13 11:53:19,769 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-top 2023-02-13 11:53:19,774 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-top 2023-02-13 11:53:19,774 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(310): Store=873f0d131aa7d6fc989e0bdac352dc63/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:19,775 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,775 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2 2023-02-13 11:53:19,776 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2 2023-02-13 11:53:19,776 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 873f0d131aa7d6fc989e0bdac352dc63 columnFamilyName cf2 2023-02-13 11:53:19,795 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-top 2023-02-13 11:53:19,801 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-top 2023-02-13 11:53:19,807 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-top 2023-02-13 11:53:19,813 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-top 2023-02-13 11:53:19,819 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-top 2023-02-13 11:53:19,825 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-top 2023-02-13 11:53:19,831 DEBUG [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-top 2023-02-13 11:53:19,831 INFO [StoreOpener-873f0d131aa7d6fc989e0bdac352dc63-1] regionserver.HStore(310): Store=873f0d131aa7d6fc989e0bdac352dc63/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:19,832 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:19,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:19,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 873f0d131aa7d6fc989e0bdac352dc63; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67782189, jitterRate=0.010033324360847473}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:19,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 873f0d131aa7d6fc989e0bdac352dc63: Running coprocessor pre-open hook at 1676289199715Writing region info on filesystem at 1676289199715Initializing all the Stores at 1676289199716 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289199716Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289199716Cleaning up temporary data from old regions at 1676289199834 (+118 ms)Cleaning up detritus from prior splits at 1676289199834Running coprocessor post-open hooks at 1676289199837 (+3 ms)Region opened successfully at 1676289199837 2023-02-13 11:53:19,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63., pid=33, masterSystemTime=1676289199709 2023-02-13 11:53:19,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:19,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:19,840 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:19,840 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:19,840 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,840 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,840 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1912): 873f0d131aa7d6fc989e0bdac352dc63/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:19,840 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1912): 873f0d131aa7d6fc989e0bdac352dc63/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:19,840 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 873f0d131aa7d6fc989e0bdac352dc63/cf2 in testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,840 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 873f0d131aa7d6fc989e0bdac352dc63/cf1 in testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,841 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/.tmp, totalSize=34.9 K 2023-02-13 11:53:19,841 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/.tmp, totalSize=34.9 K 2023-02-13 11:53:19,841 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,842 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289193826 2023-02-13 11:53:19,841 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289193826 2023-02-13 11:53:19,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:19,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:19,842 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 553d82e2c9cc5115f61d7f068d12f829, NAME => 'testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 11:53:19,842 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=873f0d131aa7d6fc989e0bdac352dc63, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:19,842 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289194132 2023-02-13 11:53:19,842 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289199842"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289199842"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289199842"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289199842"}]},"ts":"1676289199842"} 2023-02-13 11:53:19,842 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289194132 2023-02-13 11:53:19,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:19,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:19,843 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289194448 2023-02-13 11:53:19,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:19,843 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289194448 2023-02-13 11:53:19,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:19,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:19,843 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289194737 2023-02-13 11:53:19,844 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289194737 2023-02-13 11:53:19,844 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289195022 2023-02-13 11:53:19,844 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289195022 2023-02-13 11:53:19,845 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:19,845 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289195386 2023-02-13 11:53:19,845 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289195386 2023-02-13 11:53:19,846 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289195648 2023-02-13 11:53:19,846 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289195648 2023-02-13 11:53:19,847 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1 2023-02-13 11:53:19,848 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1 2023-02-13 11:53:19,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=32 2023-02-13 11:53:19,848 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 553d82e2c9cc5115f61d7f068d12f829 columnFamilyName cf1 2023-02-13 11:53:19,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=32, state=SUCCESS; OpenRegionProcedure 873f0d131aa7d6fc989e0bdac352dc63, server=jenkins-hbase16.apache.org,44985,1676289185170 in 288 msec 2023-02-13 11:53:19,850 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, ASSIGN in 450 msec 2023-02-13 11:53:19,869 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] throttle.PressureAwareThroughputController(145): 873f0d131aa7d6fc989e0bdac352dc63#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:19,874 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-bottom 2023-02-13 11:53:19,877 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] throttle.PressureAwareThroughputController(145): 873f0d131aa7d6fc989e0bdac352dc63#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:19,885 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-bottom 2023-02-13 11:53:19,892 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-bottom 2023-02-13 11:53:19,905 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-bottom 2023-02-13 11:53:19,942 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/.tmp/cf2/02afcf07e0ff4e5393f46f649fb73756 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/02afcf07e0ff4e5393f46f649fb73756 2023-02-13 11:53:19,943 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-bottom 2023-02-13 11:53:19,946 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/.tmp/cf1/10e32d241ec64155bcf738bd671fede8 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/10e32d241ec64155bcf738bd671fede8 2023-02-13 11:53:19,949 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 873f0d131aa7d6fc989e0bdac352dc63/cf2 of 873f0d131aa7d6fc989e0bdac352dc63 into 02afcf07e0ff4e5393f46f649fb73756(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:19,949 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 873f0d131aa7d6fc989e0bdac352dc63: 2023-02-13 11:53:19,949 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63., storeName=873f0d131aa7d6fc989e0bdac352dc63/cf2, priority=9, startTime=1676289199838; duration=0sec 2023-02-13 11:53:19,949 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:19,954 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-bottom 2023-02-13 11:53:19,956 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 873f0d131aa7d6fc989e0bdac352dc63/cf1 of 873f0d131aa7d6fc989e0bdac352dc63 into 10e32d241ec64155bcf738bd671fede8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:19,956 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 873f0d131aa7d6fc989e0bdac352dc63: 2023-02-13 11:53:19,956 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63., storeName=873f0d131aa7d6fc989e0bdac352dc63/cf1, priority=9, startTime=1676289199838; duration=0sec 2023-02-13 11:53:19,956 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:19,960 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-bottom 2023-02-13 11:53:19,961 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(310): Store=553d82e2c9cc5115f61d7f068d12f829/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:19,961 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:19,962 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2 2023-02-13 11:53:19,962 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2 2023-02-13 11:53:19,962 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 553d82e2c9cc5115f61d7f068d12f829 columnFamilyName cf2 2023-02-13 11:53:19,981 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-bottom 2023-02-13 11:53:19,986 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-bottom 2023-02-13 11:53:19,990 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-bottom 2023-02-13 11:53:19,995 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-bottom 2023-02-13 11:53:20,000 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-bottom 2023-02-13 11:53:20,005 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-bottom 2023-02-13 11:53:20,009 DEBUG [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-bottom 2023-02-13 11:53:20,009 INFO [StoreOpener-553d82e2c9cc5115f61d7f068d12f829-1] regionserver.HStore(310): Store=553d82e2c9cc5115f61d7f068d12f829/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:20,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:20,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 553d82e2c9cc5115f61d7f068d12f829; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65856962, jitterRate=-0.01865479350090027}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:20,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 553d82e2c9cc5115f61d7f068d12f829: Running coprocessor pre-open hook at 1676289199843Writing region info on filesystem at 1676289199844 (+1 ms)Initializing all the Stores at 1676289199844Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289199844Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289199845 (+1 ms)Cleaning up temporary data from old regions at 1676289200012 (+167 ms)Cleaning up detritus from prior splits at 1676289200013 (+1 ms)Running coprocessor post-open hooks at 1676289200015 (+2 ms)Region opened successfully at 1676289200015 2023-02-13 11:53:20,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829., pid=34, masterSystemTime=1676289199709 2023-02-13 11:53:20,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:20,017 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:20,017 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:20,017 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:20,017 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,017 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1912): 553d82e2c9cc5115f61d7f068d12f829/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:20,017 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,017 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 553d82e2c9cc5115f61d7f068d12f829/cf1 in testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,017 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1912): 553d82e2c9cc5115f61d7f068d12f829/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:20,017 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 553d82e2c9cc5115f61d7f068d12f829/cf2 in testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,018 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/.tmp, totalSize=34.9 K 2023-02-13 11:53:20,018 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/.tmp, totalSize=34.9 K 2023-02-13 11:53:20,018 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289193826 2023-02-13 11:53:20,018 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289193826 2023-02-13 11:53:20,019 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,019 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289194132 2023-02-13 11:53:20,019 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=553d82e2c9cc5115f61d7f068d12f829, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:20,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289200019"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289200019"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289200019"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289200019"}]},"ts":"1676289200019"} 2023-02-13 11:53:20,019 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289194132 2023-02-13 11:53:20,019 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289194448 2023-02-13 11:53:20,020 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289194448 2023-02-13 11:53:20,020 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289194737 2023-02-13 11:53:20,020 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289194737 2023-02-13 11:53:20,020 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289195022 2023-02-13 11:53:20,021 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289195022 2023-02-13 11:53:20,021 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289195386 2023-02-13 11:53:20,021 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting 50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289195386 2023-02-13 11:53:20,021 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting 3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289195648 2023-02-13 11:53:20,021 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289195648 2023-02-13 11:53:20,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=31 2023-02-13 11:53:20,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=31, state=SUCCESS; OpenRegionProcedure 553d82e2c9cc5115f61d7f068d12f829, server=jenkins-hbase16.apache.org,44985,1676289185170 in 459 msec 2023-02-13 11:53:20,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-02-13 11:53:20,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, ASSIGN in 625 msec 2023-02-13 11:53:20,030 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 in 4.1160 sec 2023-02-13 11:53:20,039 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] throttle.PressureAwareThroughputController(145): 553d82e2c9cc5115f61d7f068d12f829#cf2#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:20,041 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] throttle.PressureAwareThroughputController(145): 553d82e2c9cc5115f61d7f068d12f829#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:20,069 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/.tmp/cf2/7ee586fc351d4cc2a9053d6784972eeb as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/7ee586fc351d4cc2a9053d6784972eeb 2023-02-13 11:53:20,069 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/.tmp/cf1/fd3e5ff93ce648fd82853089470651f5 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/fd3e5ff93ce648fd82853089470651f5 2023-02-13 11:53:20,075 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 553d82e2c9cc5115f61d7f068d12f829/cf1 of 553d82e2c9cc5115f61d7f068d12f829 into fd3e5ff93ce648fd82853089470651f5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:20,075 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 553d82e2c9cc5115f61d7f068d12f829/cf2 of 553d82e2c9cc5115f61d7f068d12f829 into 7ee586fc351d4cc2a9053d6784972eeb(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:20,075 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 553d82e2c9cc5115f61d7f068d12f829: 2023-02-13 11:53:20,075 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 553d82e2c9cc5115f61d7f068d12f829: 2023-02-13 11:53:20,075 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829., storeName=553d82e2c9cc5115f61d7f068d12f829/cf1, priority=9, startTime=1676289200016; duration=0sec 2023-02-13 11:53:20,075 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829., storeName=553d82e2c9cc5115f61d7f068d12f829/cf2, priority=9, startTime=1676289200017; duration=0sec 2023-02-13 11:53:20,076 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:20,076 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:20,142 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44985] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 188.40.62.62:44498 deadline: 1676289260142, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de. is not online on jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:20,257 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:53:20,267 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 11:53:20,270 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:53:20,293 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:20,295 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-02-13 11:53:20,295 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitWithoutPONR 2023-02-13 11:53:20,296 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,300 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 11:53:20,300 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289200300"}]},"ts":"1676289200300"} 2023-02-13 11:53:20,306 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-02-13 11:53:20,316 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-02-13 11:53:20,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, UNASSIGN}] 2023-02-13 11:53:20,318 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, UNASSIGN 2023-02-13 11:53:20,319 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=873f0d131aa7d6fc989e0bdac352dc63, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:20,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289200319"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289200319"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289200319"}]},"ts":"1676289200319"} 2023-02-13 11:53:20,321 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 873f0d131aa7d6fc989e0bdac352dc63, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:20,324 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, UNASSIGN 2023-02-13 11:53:20,325 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=553d82e2c9cc5115f61d7f068d12f829, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:20,325 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289200325"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289200325"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289200325"}]},"ts":"1676289200325"} 2023-02-13 11:53:20,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 553d82e2c9cc5115f61d7f068d12f829, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:53:20,401 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 11:53:20,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:20,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 873f0d131aa7d6fc989e0bdac352dc63, disabling compactions & flushes 2023-02-13 11:53:20,473 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:20,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:20,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. after waiting 0 ms 2023-02-13 11:53:20,473 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:20,487 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-top] to archive 2023-02-13 11:53:20,488 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:20,490 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,492 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,493 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,494 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,495 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,497 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,498 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,500 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-top] to archive 2023-02-13 11:53:20,500 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:20,502 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,503 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,505 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,506 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,507 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,508 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,509 DEBUG [StoreCloser-testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,513 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:53:20,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:20,514 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63. 2023-02-13 11:53:20,514 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 873f0d131aa7d6fc989e0bdac352dc63: Waiting for close lock at 1676289200473Running coprocessor pre-close hooks at 1676289200473Disabling compacts and flushes for region at 1676289200473Disabling writes for close at 1676289200473Writing region close event to WAL at 1676289200510 (+37 ms)Running coprocessor post-close hooks at 1676289200514 (+4 ms)Closed at 1676289200514 2023-02-13 11:53:20,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:20,516 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 553d82e2c9cc5115f61d7f068d12f829, disabling compactions & flushes 2023-02-13 11:53:20,517 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. after waiting 0 ms 2023-02-13 11:53:20,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,517 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=873f0d131aa7d6fc989e0bdac352dc63, regionState=CLOSED 2023-02-13 11:53:20,517 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289200517"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289200517"}]},"ts":"1676289200517"} 2023-02-13 11:53:20,518 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07-bottom] to archive 2023-02-13 11:53:20,519 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:20,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-02-13 11:53:20,521 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 873f0d131aa7d6fc989e0bdac352dc63, server=jenkins-hbase16.apache.org,44985,1676289185170 in 198 msec 2023-02-13 11:53:20,522 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=873f0d131aa7d6fc989e0bdac352dc63, UNASSIGN in 204 msec 2023-02-13 11:53:20,523 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,525 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,526 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,527 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,528 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,529 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,531 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5-bottom] to archive 2023-02-13 11:53:20,532 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:20,533 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,534 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,535 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,536 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,537 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,538 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,539 DEBUG [StoreCloser-testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,543 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:53:20,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:20,544 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829. 2023-02-13 11:53:20,544 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 553d82e2c9cc5115f61d7f068d12f829: Waiting for close lock at 1676289200517Running coprocessor pre-close hooks at 1676289200517Disabling compacts and flushes for region at 1676289200517Disabling writes for close at 1676289200517Writing region close event to WAL at 1676289200540 (+23 ms)Running coprocessor post-close hooks at 1676289200544 (+4 ms)Closed at 1676289200544 2023-02-13 11:53:20,545 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,546 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=553d82e2c9cc5115f61d7f068d12f829, regionState=CLOSED 2023-02-13 11:53:20,546 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289200546"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289200546"}]},"ts":"1676289200546"} 2023-02-13 11:53:20,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-02-13 11:53:20,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 553d82e2c9cc5115f61d7f068d12f829, server=jenkins-hbase16.apache.org,44985,1676289185170 in 220 msec 2023-02-13 11:53:20,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=36, resume processing ppid=35 2023-02-13 11:53:20,551 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=553d82e2c9cc5115f61d7f068d12f829, UNASSIGN in 232 msec 2023-02-13 11:53:20,552 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289200552"}]},"ts":"1676289200552"} 2023-02-13 11:53:20,553 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-02-13 11:53:20,560 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-02-13 11:53:20,564 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 266 msec 2023-02-13 11:53:20,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-02-13 11:53:20,603 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-02-13 11:53:20,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitWithoutPONR 2023-02-13 11:53:20,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,608 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,609 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,609 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 11:53:20,617 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,617 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,617 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:20,621 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/recovered.edits] 2023-02-13 11:53:20,621 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/recovered.edits] 2023-02-13 11:53:20,621 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/recovered.edits] 2023-02-13 11:53:20,626 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/.regioninfo to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/.regioninfo 2023-02-13 11:53:20,626 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/.regioninfo to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/.regioninfo 2023-02-13 11:53:20,627 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/352a24241fb04817b9724f9a0766b80d 2023-02-13 11:53:20,630 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/376a34289efe48bd8b6cf6a48becdcd7 2023-02-13 11:53:20,631 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,632 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/352a24241fb04817b9724f9a0766b80d.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,632 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/50de1a674b394cff96414e041962053f 2023-02-13 11:53:20,633 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,633 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/376a34289efe48bd8b6cf6a48becdcd7.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,633 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/676353ee928f426cb1a9ee45791c521a 2023-02-13 11:53:20,635 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,636 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/50de1a674b394cff96414e041962053f.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,636 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/8ee354f1470e40599fb318ba0a2205e0 2023-02-13 11:53:20,636 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,637 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07 2023-02-13 11:53:20,637 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/676353ee928f426cb1a9ee45791c521a.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,638 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,639 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf1/c70362f983a14f8ca6bc19efc048546e 2023-02-13 11:53:20,639 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/8ee354f1470e40599fb318ba0a2205e0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,639 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,640 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/a5a8cc605c4d4b10a7f6d53bcf969e07.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,641 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,642 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf1/c70362f983a14f8ca6bc19efc048546e.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,643 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/13baf99673e045eebc4a13bc9f309c8c 2023-02-13 11:53:20,644 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,644 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5 2023-02-13 11:53:20,646 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/13baf99673e045eebc4a13bc9f309c8c.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,647 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/67664b1791514f50894e6ebbc320a3b0 2023-02-13 11:53:20,647 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,649 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/3c46c512b0b24cc3a2b8f75592ec3ac5.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,649 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,649 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/98bfdba0aa8e40e29830fa62b2318f21 2023-02-13 11:53:20,650 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/67664b1791514f50894e6ebbc320a3b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,651 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,651 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/c902e027cb294e428fc0ce1c657c13b3 2023-02-13 11:53:20,652 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/98bfdba0aa8e40e29830fa62b2318f21.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,653 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,653 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0 2023-02-13 11:53:20,654 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/c902e027cb294e428fc0ce1c657c13b3.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,655 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,655 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/cf2/f7e1d14bb6114288aece80770e5cbee8 2023-02-13 11:53:20,657 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/873f0d131aa7d6fc989e0bdac352dc63/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,657 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/ce4b5f71aef4410aaf2e61ba81fdb5b0.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,659 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/553d82e2c9cc5115f61d7f068d12f829/cf2/f7e1d14bb6114288aece80770e5cbee8.00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,659 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de/recovered.edits/85.seqid 2023-02-13 11:53:20,660 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/10e32d241ec64155bcf738bd671fede8 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf1/10e32d241ec64155bcf738bd671fede8 2023-02-13 11:53:20,661 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/00e2700a0258d2e58039799b308648de 2023-02-13 11:53:20,662 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/fd3e5ff93ce648fd82853089470651f5 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf1/fd3e5ff93ce648fd82853089470651f5 2023-02-13 11:53:20,664 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/02afcf07e0ff4e5393f46f649fb73756 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/cf2/02afcf07e0ff4e5393f46f649fb73756 2023-02-13 11:53:20,670 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/7ee586fc351d4cc2a9053d6784972eeb to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/cf2/7ee586fc351d4cc2a9053d6784972eeb 2023-02-13 11:53:20,671 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63/recovered.edits/90.seqid 2023-02-13 11:53:20,674 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:20,676 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829/recovered.edits/90.seqid 2023-02-13 11:53:20,677 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitWithoutPONR/553d82e2c9cc5115f61d7f068d12f829 2023-02-13 11:53:20,677 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-02-13 11:53:20,680 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,682 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-02-13 11:53:20,685 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-02-13 11:53:20,686 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,686 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-02-13 11:53:20,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289200686"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:20,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289200686"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:20,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289200686"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:20,689 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:53:20,689 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 00e2700a0258d2e58039799b308648de, NAME => 'testSplitWithoutPONR,,1676289193316.00e2700a0258d2e58039799b308648de.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 553d82e2c9cc5115f61d7f068d12f829, NAME => 'testSplitWithoutPONR,,1676289195912.553d82e2c9cc5115f61d7f068d12f829.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 873f0d131aa7d6fc989e0bdac352dc63, NAME => 'testSplitWithoutPONR,41,1676289195912.873f0d131aa7d6fc989e0bdac352dc63.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 11:53:20,689 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-02-13 11:53:20,689 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289200689"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:20,692 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-02-13 11:53:20,703 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:20,705 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 99 msec 2023-02-13 11:53:20,711 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-02-13 11:53:20,711 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-02-13 11:53:20,728 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=404 (was 392) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:37420 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:44766 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:44792 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:37454 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:53354 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:55446 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:37432 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:37456 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:54484 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:59708 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:60462 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase16:44985-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:59690 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-3 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:59688 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS-EventLoopGroup-4-2 org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native Method) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:209) org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.epollWait(Native.java:202) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.epollWaitNoTimerChange(EpollEventLoop.java:306) org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:363) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:59698 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:44768 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1044005244_15 at /127.0.0.1:37468 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=871 (was 851) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=300 (was 309), ProcessCount=169 (was 169), AvailableMemoryMB=5409 (was 6283) 2023-02-13 11:53:20,749 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=404, OpenFileDescriptor=871, MaxFileDescriptor=60000, SystemLoadAverage=300, ProcessCount=169, AvailableMemoryMB=5408 2023-02-13 11:53:20,750 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:20,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:20,761 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:20,762 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:20,765 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:20,766 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:20,768 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:20,768 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 empty. 2023-02-13 11:53:20,769 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:20,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 11:53:20,790 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:20,791 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => 5249cf8a85fb696a935f1c38070bb429, NAME => 'testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing 5249cf8a85fb696a935f1c38070bb429, disabling compactions & flushes 2023-02-13 11:53:20,807 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. after waiting 0 ms 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:20,807 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:20,807 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for 5249cf8a85fb696a935f1c38070bb429: Waiting for close lock at 1676289200807Disabling compacts and flushes for region at 1676289200807Disabling writes for close at 1676289200807Writing region close event to WAL at 1676289200807Closed at 1676289200807 2023-02-13 11:53:20,810 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:20,811 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289200810"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289200810"}]},"ts":"1676289200810"} 2023-02-13 11:53:20,812 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:20,814 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:20,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289200814"}]},"ts":"1676289200814"} 2023-02-13 11:53:20,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 11:53:20,872 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:20,873 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:20,873 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:20,873 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:20,873 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:20,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN}] 2023-02-13 11:53:20,876 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:20,877 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:53:21,027 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:21,029 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289201028"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289201028"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289201028"}]},"ts":"1676289201028"} 2023-02-13 11:53:21,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:21,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5249cf8a85fb696a935f1c38070bb429, NAME => 'testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:21,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,190 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,192 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,193 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1 2023-02-13 11:53:21,193 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1 2023-02-13 11:53:21,193 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5249cf8a85fb696a935f1c38070bb429 columnFamilyName cf1 2023-02-13 11:53:21,194 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(310): Store=5249cf8a85fb696a935f1c38070bb429/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:21,194 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,195 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2 2023-02-13 11:53:21,195 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2 2023-02-13 11:53:21,196 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5249cf8a85fb696a935f1c38070bb429 columnFamilyName cf2 2023-02-13 11:53:21,196 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(310): Store=5249cf8a85fb696a935f1c38070bb429/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:21,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:21,201 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:21,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5249cf8a85fb696a935f1c38070bb429; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65024061, jitterRate=-0.031065985560417175}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:21,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5249cf8a85fb696a935f1c38070bb429: Running coprocessor pre-open hook at 1676289201191Writing region info on filesystem at 1676289201191Initializing all the Stores at 1676289201191Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289201191Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289201192 (+1 ms)Cleaning up temporary data from old regions at 1676289201198 (+6 ms)Cleaning up detritus from prior splits at 1676289201199 (+1 ms)Running coprocessor post-open hooks at 1676289201203 (+4 ms)Region opened successfully at 1676289201203 2023-02-13 11:53:21,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429., pid=43, masterSystemTime=1676289201182 2023-02-13 11:53:21,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289201206"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289201206"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289201206"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289201206"}]},"ts":"1676289201206"} 2023-02-13 11:53:21,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-02-13 11:53:21,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 in 177 msec 2023-02-13 11:53:21,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-02-13 11:53:21,211 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN in 335 msec 2023-02-13 11:53:21,212 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:21,212 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289201212"}]},"ts":"1676289201212"} 2023-02-13 11:53:21,213 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 11:53:21,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:21,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 458 msec 2023-02-13 11:53:21,265 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-02-13 11:53:21,269 INFO [RS-EventLoopGroup-5-2] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:43216, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-02-13 11:53:21,277 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:21,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:21,279 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:21,279 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,279 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:21,279 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:21,280 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,280 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,285 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,286 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,287 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,288 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,288 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,289 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,289 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:21,290 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,290 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:21,290 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,290 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:21,290 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,290 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,291 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,291 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:53:21,291 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,294 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,302 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,302 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,302 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,302 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,302 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,321 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/6b4fd7e2853f4617834a0d178456c990 2023-02-13 11:53:21,344 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/0f6d684af4f54f40b9c588aa4e8c0413 2023-02-13 11:53:21,352 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/6b4fd7e2853f4617834a0d178456c990 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 2023-02-13 11:53:21,357 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:21,358 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/0f6d684af4f54f40b9c588aa4e8c0413 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 2023-02-13 11:53:21,364 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:21,365 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 74ms, sequenceid=6, compaction requested=false 2023-02-13 11:53:21,365 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-02-13 11:53:21,366 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:21,366 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,366 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:21,366 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:21,366 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,367 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,367 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,377 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,377 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:21,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:21,378 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,378 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:21,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:21,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:21,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@38b118f4[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:21,380 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:21,380 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,392 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,392 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:21,391 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:21,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:21,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:21,392 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:21,402 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:21,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:21,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:21,402 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:21,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:21,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:21,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:21,411 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:21,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:21,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:21,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:21,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:21,414 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:21,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:21,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:21,414 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:21,414 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:21,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:21,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:21,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,425 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,424 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:21,425 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,425 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,428 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:21,429 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:21,430 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,430 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,431 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,442 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:21,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:21,442 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,442 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:21,442 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,441 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,442 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,444 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:21,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:21,544 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:21,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:21,573 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:21,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:21,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:21,580 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:21,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:21,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:21,581 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,581 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,635 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,635 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,635 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,635 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:21,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,636 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:21,636 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,636 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:21,638 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:21,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:21,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,638 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:21,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,638 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,638 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:21,639 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:21,639 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,639 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:21,639 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:21,677 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/ce063ca9705c495b87f539ed8b997f3a 2023-02-13 11:53:21,693 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/c84a48c1b00540939eec2407b845d087 2023-02-13 11:53:21,697 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/ce063ca9705c495b87f539ed8b997f3a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a 2023-02-13 11:53:21,703 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:21,704 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/c84a48c1b00540939eec2407b845d087 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 2023-02-13 11:53:21,710 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:21,711 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 72ms, sequenceid=19, compaction requested=false 2023-02-13 11:53:21,711 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:21,711 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:21,711 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:21,711 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:21,711 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,711 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:21,711 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:21,735 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,810 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,810 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,810 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,866 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:21,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:21,866 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,867 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,867 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:21,867 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:21,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:21,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:21,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:21,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:21,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:21,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:21,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:21,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:21,871 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72bbeacd[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:21,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:21,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,033 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,033 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,033 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,034 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:22,034 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:22,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,036 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:22,035 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:22,116 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,116 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,117 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,117 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,124 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,124 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,125 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,141 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,141 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,141 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:22,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:22,144 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:22,145 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:22,145 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:22,145 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:22,145 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,153 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,153 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,153 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,154 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,154 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,155 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,156 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,156 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,156 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,156 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,166 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:22,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,167 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:22,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,168 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,268 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:22,269 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:22,284 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:22,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:22,285 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:22,285 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:22,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:22,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,286 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,294 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,294 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,294 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,294 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,294 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,295 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,295 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,296 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,295 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,296 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,296 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,296 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:22,296 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:22,302 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,307 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/aff3a737759a4346b37626329f50fca0 2023-02-13 11:53:22,310 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,310 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,310 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,311 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,311 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,322 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/d006d5a631e645efbabe9871463bd806 2023-02-13 11:53:22,326 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/aff3a737759a4346b37626329f50fca0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 2023-02-13 11:53:22,331 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:22,332 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/d006d5a631e645efbabe9871463bd806 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 2023-02-13 11:53:22,337 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:22,338 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 42ms, sequenceid=32, compaction requested=true 2023-02-13 11:53:22,338 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:22,338 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,338 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:22,338 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:22,338 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,338 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,338 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,349 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,349 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,350 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,350 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:22,352 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@16110508[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:22,352 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:22,352 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,360 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,360 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,360 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:22,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,361 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:22,361 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,362 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:22,369 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,369 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,380 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:22,380 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:22,380 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:22,381 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:22,381 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:22,381 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:22,381 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,391 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,392 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,393 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,393 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,397 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,397 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,398 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,429 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,430 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:22,429 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,429 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:22,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,430 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,431 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,530 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:22,531 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:22,548 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:22,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:22,550 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:22,550 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:22,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:22,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,560 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,560 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,562 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,563 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,563 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,563 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,563 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,563 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,563 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,564 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,564 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,564 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,564 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,564 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,564 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,564 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,567 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,567 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,567 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,568 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,568 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,568 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:22,568 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:22,568 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,568 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:22,568 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:22,569 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,569 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,569 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,577 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,577 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,577 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,579 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/bfc806e60fe7474db07acb9a62bf9b6e 2023-02-13 11:53:22,594 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/ace72e50efed4cd1bc51cdd7030c153c 2023-02-13 11:53:22,599 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/bfc806e60fe7474db07acb9a62bf9b6e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e 2023-02-13 11:53:22,604 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:22,605 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/ace72e50efed4cd1bc51cdd7030c153c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c 2023-02-13 11:53:22,610 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:22,611 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 43ms, sequenceid=45, compaction requested=true 2023-02-13 11:53:22,611 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:22,611 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,611 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:22,611 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:22,611 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,611 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,611 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,619 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,619 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,619 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,619 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:22,621 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@427ea0d7[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:22,621 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:22,621 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,627 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,627 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,627 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,628 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:22,628 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,628 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,641 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:22,644 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:22,644 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:22,644 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:22,644 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:22,644 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:22,644 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,658 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,659 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,659 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,660 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,661 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,661 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,669 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:22,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:22,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,669 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:22,670 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,670 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,670 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,770 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:22,771 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:22,773 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:22,800 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:22,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:22,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:22,802 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,802 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,802 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:22,802 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:22,803 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,803 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,812 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,812 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,812 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,813 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,813 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,814 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:22,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,814 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:22,814 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,814 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,814 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:22,815 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,815 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,816 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:22,816 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:22,816 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,816 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:22,816 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:22,824 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,833 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,833 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,833 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,833 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,833 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,838 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/5dfaaebc8604485ebc13056a00bf3b10 2023-02-13 11:53:22,852 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/c8208bc7abca4a3fa622fe266751c97a 2023-02-13 11:53:22,857 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/5dfaaebc8604485ebc13056a00bf3b10 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 2023-02-13 11:53:22,862 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:22,863 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/c8208bc7abca4a3fa622fe266751c97a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a 2023-02-13 11:53:22,867 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:22,868 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 52ms, sequenceid=58, compaction requested=true 2023-02-13 11:53:22,868 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:22,868 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:22,868 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:22,868 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:22,868 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,868 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:22,868 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:22,883 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,883 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,883 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,883 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:22,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:22,887 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72b866d4[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:22,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:22,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:22,894 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,894 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:22,894 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:22,922 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,922 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,922 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:22,936 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,936 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,935 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,936 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:22,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:22,947 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:22,947 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:22,948 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:22,948 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:22,948 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:22,948 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:22,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,959 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:22,959 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,959 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:22,960 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,960 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,960 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:22,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:22,961 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:22,961 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,962 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,969 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:22,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:22,969 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:22,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:22,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:22,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:22,969 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:22,970 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,970 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:22,970 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,970 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:22,970 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:22,970 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,070 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:23,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:23,088 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:23,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:23,090 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-02-13 11:53:23,090 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-02-13 11:53:23,090 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:23,090 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-02-13 11:53:23,090 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:53:23,091 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,091 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,099 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,099 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,100 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,100 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,100 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:23,101 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:23,101 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:23,101 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:23,101 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:23,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:23,102 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:23,102 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:23,102 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:23,102 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. started... 2023-02-13 11:53:23,103 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:23,110 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,110 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,111 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,111 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:23,111 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,111 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:23,114 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/54e71050b34e4772895f463aff5b16be 2023-02-13 11:53:23,130 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/3505ceea0e064689a125b46cdf34c4eb 2023-02-13 11:53:23,134 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/54e71050b34e4772895f463aff5b16be as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be 2023-02-13 11:53:23,140 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:23,141 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/3505ceea0e064689a125b46cdf34c4eb as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb 2023-02-13 11:53:23,145 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:23,146 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 43ms, sequenceid=71, compaction requested=true 2023-02-13 11:53:23,146 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:23,146 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-02-13 11:53:23,146 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-02-13 11:53:23,152 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,152 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:23,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:23,152 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:23,152 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:23,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:23,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,153 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,154 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:23,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-02-13 11:53:23,154 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5ff817cc[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:23,154 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:53:23,154 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,161 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,161 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:23,160 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:23,161 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-02-13 11:53:23,161 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:23,169 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:23,169 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:23,169 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:23,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:23,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:23,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:23,185 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-02-13 11:53:23,185 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,185 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:23,185 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-02-13 11:53:23,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:23,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:23,188 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:53:23,188 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-02-13 11:53:23,188 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:23,188 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:23,188 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-02-13 11:53:23,188 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,199 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,200 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,200 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,202 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,202 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:23,202 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-02-13 11:53:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,203 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,231 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,231 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-02-13 11:53:23,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:23,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:23,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:23,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:23,232 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:23,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:23,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:23,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:23,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-02-13 11:53:23,233 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-02-13 11:53:23,333 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:23,334 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:23,601 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 11:53:23,602 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:23,603 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,604 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:23,605 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:23,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:23,853 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,854 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:23,854 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:23,854 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:23,854 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-02-13 11:53:23,855 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:23,855 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:23,855 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:23,855 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:23,856 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:23,859 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 11:53:23,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:23,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:23,861 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:23,861 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:23,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:23,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:23,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:23,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:23,864 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:23,864 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:23,864 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:23,864 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:23,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:23,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,865 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:23,865 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,865 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,865 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,865 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 11:53:23,866 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:23,866 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:23,867 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289233867 2023-02-13 11:53:23,868 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,868 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:23,871 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:23,871 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:23,871 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:23,871 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289263871 2023-02-13 11:53:23,871 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289323871 2023-02-13 11:53:23,871 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:23,872 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:23,873 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:23,873 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:23,873 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:23,873 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:24,122 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:24,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:24,123 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:24,124 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 11:53:24,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:24,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:24,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:24,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:24,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:24,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,128 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:24,128 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,128 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,129 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:24,129 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,129 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,130 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,130 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,130 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,130 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:53:24,130 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:24,131 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:24,131 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289234131 2023-02-13 11:53:24,133 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:24,133 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,136 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,136 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,136 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:24,136 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289264136 2023-02-13 11:53:24,136 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289324136 2023-02-13 11:53:24,137 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:24,137 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:24,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN}] 2023-02-13 11:53:24,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:24,139 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:24,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:24,387 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:24,387 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:24,388 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:24,389 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 11:53:24,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:24,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:24,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:24,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:24,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:24,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:24,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,393 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:24,393 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,393 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,393 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,393 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,393 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,393 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 11:53:24,394 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:24,395 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:24,395 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289234395 2023-02-13 11:53:24,396 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:24,397 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,399 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,400 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,400 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:24,400 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289264400 2023-02-13 11:53:24,400 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289324400 2023-02-13 11:53:24,400 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:24,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN}] 2023-02-13 11:53:24,400 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:24,403 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,404 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,404 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289204404"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289204404"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289204404"}]},"ts":"1676289204404"} 2023-02-13 11:53:24,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:24,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:24,406 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:24,406 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:24,650 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,651 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:24,651 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:24,651 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:24,651 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:24,651 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:24,652 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:24,652 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:24,652 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:24,653 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:24,655 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:24,655 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:24,656 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:24,656 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:24,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:24,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:24,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:24,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:24,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:24,660 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,660 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,660 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,660 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,661 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,661 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:24,661 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:24,662 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:24,662 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:24,662 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289234662 2023-02-13 11:53:24,663 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:24,663 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,665 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,665 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSING, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,665 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:24,665 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289264665 2023-02-13 11:53:24,665 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289324665 2023-02-13 11:53:24,665 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:24,666 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,666 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289204665"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289204665"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289204665"}]},"ts":"1676289204665"} 2023-02-13 11:53:24,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:24,667 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:24,820 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:24,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5249cf8a85fb696a935f1c38070bb429, disabling compactions & flushes 2023-02-13 11:53:24,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:24,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:24,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. after waiting 0 ms 2023-02-13 11:53:24,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:24,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 5249cf8a85fb696a935f1c38070bb429 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:53:24,835 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/bd4f2bff57d04d3eb83e428e35483a07 2023-02-13 11:53:24,856 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/82b51dfe6c894a07915cbfb80066fb30 2023-02-13 11:53:24,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/bd4f2bff57d04d3eb83e428e35483a07 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 2023-02-13 11:53:24,871 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:24,872 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/82b51dfe6c894a07915cbfb80066fb30 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 2023-02-13 11:53:24,878 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:53:24,879 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 58ms, sequenceid=82, compaction requested=true 2023-02-13 11:53:24,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:53:24,884 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:24,885 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:24,885 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5249cf8a85fb696a935f1c38070bb429: Waiting for close lock at 1676289204821Running coprocessor pre-close hooks at 1676289204821Disabling compacts and flushes for region at 1676289204821Disabling writes for close at 1676289204821Obtaining lock to block concurrent updates at 1676289204821Preparing flush snapshotting stores in 5249cf8a85fb696a935f1c38070bb429 at 1676289204821Finished memstore snapshotting testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289204821Flushing stores of testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. at 1676289204822 (+1 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf1: creating writer at 1676289204823 (+1 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf1: appending metadata at 1676289204826 (+3 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf1: closing flushed file at 1676289204826Flushing 5249cf8a85fb696a935f1c38070bb429/cf2: creating writer at 1676289204844 (+18 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf2: appending metadata at 1676289204847 (+3 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf2: closing flushed file at 1676289204847Flushing 5249cf8a85fb696a935f1c38070bb429/cf1: reopening flushed file at 1676289204864 (+17 ms)Flushing 5249cf8a85fb696a935f1c38070bb429/cf2: reopening flushed file at 1676289204873 (+9 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 5249cf8a85fb696a935f1c38070bb429 in 58ms, sequenceid=82, compaction requested=true at 1676289204879 (+6 ms)Writing region close event to WAL at 1676289204880 (+1 ms)Running coprocessor post-close hooks at 1676289204884 (+4 ms)Closed at 1676289204885 (+1 ms) 2023-02-13 11:53:24,887 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:24,887 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED 2023-02-13 11:53:24,888 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289204887"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289204887"}]},"ts":"1676289204887"} 2023-02-13 11:53:24,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:24,889 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,889 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:24,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:24,916 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:24,916 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:24,917 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:24,918 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:24,918 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 11:53:24,918 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-02-13 11:53:24,918 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:24,919 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:24,919 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:24,919 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:24,919 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,919 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:24,920 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,920 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:24,920 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,920 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:24,921 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,921 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:24,921 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,922 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,922 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,922 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,922 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,922 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:24,922 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-02-13 11:53:24,923 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:24,925 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:24,926 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:24,931 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289234931 2023-02-13 11:53:24,931 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:24,932 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,935 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,935 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:24,935 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:24,935 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289264935 2023-02-13 11:53:24,935 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289324935 2023-02-13 11:53:24,935 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:24,935 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:24,936 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289204935"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289204935"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289204935"}]},"ts":"1676289204935"} 2023-02-13 11:53:24,937 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:24,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-02-13 11:53:24,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 in 270 msec 2023-02-13 11:53:24,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:24,944 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:24,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:24,944 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:25,185 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,186 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:25,186 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:25,186 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:25,189 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:25,189 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:25,189 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:25,189 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:25,189 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:25,190 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:25,191 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:25,192 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-02-13 11:53:25,192 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:25,193 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:25,193 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:25,194 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:25,194 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:25,195 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,195 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:25,195 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,196 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,196 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,197 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:25,197 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,197 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,197 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:25,198 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,198 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:25,199 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:25,199 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:25,199 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,199 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,199 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,199 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,199 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,199 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-02-13 11:53:25,200 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:25,200 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:25,200 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:25,204 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289235204 2023-02-13 11:53:25,204 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:25,205 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:25,207 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,207 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,207 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:25,207 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289265207 2023-02-13 11:53:25,207 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289325207 2023-02-13 11:53:25,207 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:25,208 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:25,209 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:25,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-02-13 11:53:25,212 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN in 809 msec 2023-02-13 11:53:25,216 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:25,216 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:25,216 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:25,458 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,458 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:25,458 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:25,458 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:25,459 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:25,459 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:25,459 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:25,459 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:25,459 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:25,461 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:25,463 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:25,463 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:25,463 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:25,464 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:25,464 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:25,465 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:25,465 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:25,466 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,466 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:25,467 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,467 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,467 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,468 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:25,468 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,468 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,468 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,469 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,469 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:25,470 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:25,470 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:25,471 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289235471 2023-02-13 11:53:25,472 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:25,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,474 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,474 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,474 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:25,474 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289265474 2023-02-13 11:53:25,474 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289325474 2023-02-13 11:53:25,474 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:25,474 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,475 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,479 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:25,487 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=5249cf8a85fb696a935f1c38070bb429, threads=14 2023-02-13 11:53:25,488 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,488 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,488 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,488 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,489 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,490 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,491 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,491 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,692 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,692 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,692 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,693 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,703 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,704 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,712 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,712 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,712 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,713 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,728 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,735 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,735 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,737 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:25,738 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 5249cf8a85fb696a935f1c38070bb429 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:25,776 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:25,776 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:25,776 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:25,975 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,975 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:25,975 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:25,975 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:25,976 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:25,976 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:25,977 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:25,977 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:25,977 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:25,979 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:25,982 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 11:53:25,982 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:25,983 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:25,983 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:25,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:25,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:25,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:25,984 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,985 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:25,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:25,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:25,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:25,986 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:25,987 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,987 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,987 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,987 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,987 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:25,987 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:25,987 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:25,988 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:25,988 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289235988 2023-02-13 11:53:25,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:25,989 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:25,992 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,992 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:25,992 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:25,992 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289265992 2023-02-13 11:53:25,992 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289325992 2023-02-13 11:53:25,992 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:26,000 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=5249cf8a85fb696a935f1c38070bb429, threads=14 2023-02-13 11:53:26,001 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,001 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,001 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,002 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,003 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,003 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,124 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,152 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,169 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,177 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,212 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,220 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,224 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,224 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,225 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,225 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,230 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,230 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,233 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,239 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 for region: 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:26,239 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 5249cf8a85fb696a935f1c38070bb429 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:26,281 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:26,288 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/284fe8700b872f36efe1864a6aa81d69/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:26,290 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/1e22a103db85da411d632256c97c4c4a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:53:26,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:26,290 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,290 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:26,290 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:26,492 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,493 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:26,493 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:26,493 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:26,494 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:26,494 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:26,495 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:26,495 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:26,495 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:26,498 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 11:53:26,501 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 11:53:26,501 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:26,501 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:26,502 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:26,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:26,503 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:26,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:26,504 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:26,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,505 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,506 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:26,506 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,506 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,506 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:26,507 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,507 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:26,507 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:26,509 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:26,509 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,509 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,510 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,510 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:26,510 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:26,510 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 11:53:26,511 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:26,511 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:26,512 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289236511 2023-02-13 11:53:26,512 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:26,513 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,515 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:26,515 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:26,516 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:26,516 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289266516 2023-02-13 11:53:26,516 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289326516 2023-02-13 11:53:26,516 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:53:26,518 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:26,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:26,521 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,521 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:26,521 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:26,767 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,768 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:26,768 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:26,768 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:26,769 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:26,769 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:26,769 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:26,769 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:26,770 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:26,771 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:26,774 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 11:53:26,774 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:26,775 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:26,775 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:26,776 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:26,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:26,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:26,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:26,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:26,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:26,780 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,780 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,780 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:26,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,781 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 9 msec 2023-02-13 11:53:26,781 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:26,782 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:26,782 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289236782 2023-02-13 11:53:26,782 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:26,785 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:26,787 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:26,787 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:26,787 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:26,787 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289266787 2023-02-13 11:53:26,787 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289326787 2023-02-13 11:53:26,787 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 11:53:26,788 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:26,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:26,799 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:26,799 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:26,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:27,040 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,040 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:27,040 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:27,041 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:27,041 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:27,042 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:27,042 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:27,042 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:27,042 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:27,044 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 2 msec 2023-02-13 11:53:27,048 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 11:53:27,048 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:27,048 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:27,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:27,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:27,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:27,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:27,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:27,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:27,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:27,052 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,052 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:27,052 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:27,052 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:27,052 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,052 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,052 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,052 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,053 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,053 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:27,053 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:27,054 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:27,054 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289237054 2023-02-13 11:53:27,054 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:27,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,059 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,059 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,059 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:27,059 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289267059 2023-02-13 11:53:27,059 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289327059 2023-02-13 11:53:27,060 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 11:53:27,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:27,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:27,063 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:27,063 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:27,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:27,311 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,311 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:27,311 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:27,311 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:27,312 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:27,312 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:27,312 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:27,312 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:27,312 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:27,314 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:27,315 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 11:53:27,315 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:27,315 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:27,316 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:27,316 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:27,317 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:27,317 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:27,318 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,319 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:27,319 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,319 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,320 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,321 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:27,321 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,321 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,322 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:27,323 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,323 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:27,324 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:27,324 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:27,324 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,324 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,324 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,324 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,324 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,324 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-02-13 11:53:27,325 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:27,325 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:27,326 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289237326 2023-02-13 11:53:27,327 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:27,327 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,329 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,329 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,329 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:27,330 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289267330 2023-02-13 11:53:27,330 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289327330 2023-02-13 11:53:27,330 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:27,330 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:27,332 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:27,332 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:27,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:27,332 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:27,580 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,580 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:27,580 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:27,580 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:27,581 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:27,581 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:27,581 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:27,581 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:27,581 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:27,583 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:27,585 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 11:53:27,585 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:27,585 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:27,585 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:27,586 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:27,587 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:27,587 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:27,587 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,588 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:27,588 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,589 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,589 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,590 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:27,590 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,591 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,591 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:27,592 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,593 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:27,593 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:27,593 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:27,593 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,593 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,594 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,594 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,594 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,594 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-02-13 11:53:27,595 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:27,596 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:27,596 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289237596 2023-02-13 11:53:27,598 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,598 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:27,600 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,600 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,600 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:27,600 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289267600 2023-02-13 11:53:27,600 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289327600 2023-02-13 11:53:27,600 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:53:27,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:27,604 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN] 2023-02-13 11:53:27,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:27,605 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:27,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:27,605 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:27,851 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,851 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:27,851 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:27,851 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:27,853 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:27,853 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:27,853 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:27,854 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:27,854 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:27,855 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-02-13 11:53:27,857 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 11:53:27,857 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:53:27,857 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:53:27,857 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:27,858 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:27,858 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:27,859 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:27,859 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:27,859 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,859 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:27,859 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:27,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:27,860 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,861 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:27,861 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:27,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:27,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:27,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:27,862 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:27,862 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,862 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,862 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,863 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:27,863 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:53:27,864 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:27,864 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:27,864 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:27,865 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289237865 2023-02-13 11:53:27,867 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:27,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:27,869 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:27,871 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,871 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:27,871 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:27,871 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289267871 2023-02-13 11:53:27,871 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289327871 2023-02-13 11:53:27,871 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:27,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:27,876 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:27,876 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:27,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:27,876 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:28,121 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,121 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:28,122 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:28,122 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:28,123 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:28,123 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:53:28,124 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:28,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:28,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:28,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:28,124 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:28,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:28,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,125 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:28,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,126 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:28,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:28,127 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,127 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,127 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,127 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:28,127 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 11:53:28,128 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:28,129 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:28,129 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:28,130 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289238130 2023-02-13 11:53:28,131 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:28,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,134 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,136 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,136 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,136 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:28,136 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289268136 2023-02-13 11:53:28,137 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289328137 2023-02-13 11:53:28,137 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 5 msec 2023-02-13 11:53:28,137 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:28,140 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:28,140 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:28,140 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:28,140 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:28,387 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,387 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:28,387 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:28,387 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:28,387 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:28,388 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:28,388 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:28,388 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:28,388 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:28,389 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:28,389 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 11:53:28,389 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:28,389 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:28,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:28,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:28,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:28,390 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:28,391 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:28,392 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,392 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,392 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,392 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:28,393 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-02-13 11:53:28,393 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:28,394 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:28,394 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:28,395 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289238395 2023-02-13 11:53:28,395 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:28,397 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,398 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,399 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,400 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,400 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:28,400 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289268400 2023-02-13 11:53:28,400 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289328400 2023-02-13 11:53:28,400 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:28,400 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:28,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:28,402 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-02-13 11:53:28,402 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:28,402 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:28,650 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,650 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:28,651 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:28,651 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:28,652 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:28,652 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:28,652 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:28,652 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:28,652 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:28,653 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:28,654 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 11:53:28,654 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-02-13 11:53:28,655 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:28,655 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:28,655 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:28,655 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:28,656 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,656 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:28,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,657 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:28,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,658 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:28,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,659 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:28,659 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,660 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:53:28,660 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:53:28,662 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:28,663 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:28,663 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:28,665 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289238665 2023-02-13 11:53:28,667 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,667 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:28,671 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,673 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,674 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,674 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:28,674 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289268674 2023-02-13 11:53:28,674 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289328674 2023-02-13 11:53:28,674 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 11:53:28,675 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:53:28,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a exec-time=5.0730 sec 2023-02-13 11:53:28,679 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,680 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=true 2023-02-13 11:53:28,830 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:28,831 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:28,832 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289208831"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289208831"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289208831"}]},"ts":"1676289208831"} 2023-02-13 11:53:28,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:28,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:53:28,835 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,835 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:53:28,835 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:53:28,925 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:28,925 WARN [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:53:28,925 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:53:28,925 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:53:28,926 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:53:28,926 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:53:28,927 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-02-13 11:53:28,928 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:53:28,928 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:53:28,929 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:53:28,929 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:53:28,929 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,930 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:53:28,930 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,930 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:53:28,931 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,931 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:53:28,931 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,931 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:53:28,931 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:28,932 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:53:28,932 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,932 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:53:28,932 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,933 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:53:28,933 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:53:28,934 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:53:28,934 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429 to restore RIT 2023-02-13 11:53:28,935 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289238935 2023-02-13 11:53:28,935 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:53:28,936 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:53:28,938 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,939 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=5249cf8a85fb696a935f1c38070bb429, regionState=OPENING, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:53:28,939 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:53:28,939 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289268939 2023-02-13 11:53:28,939 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289328939 2023-02-13 11:53:28,939 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 4 msec 2023-02-13 11:53:28,940 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:28,940 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289208939"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289208939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289208939"}]},"ts":"1676289208939"} 2023-02-13 11:53:28,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:29,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,098 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 5249cf8a85fb696a935f1c38070bb429, NAME => 'testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:29,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:29,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:29,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,112 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,114 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1 2023-02-13 11:53:29,114 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1 2023-02-13 11:53:29,114 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5249cf8a85fb696a935f1c38070bb429 columnFamilyName cf1 2023-02-13 11:53:29,122 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be 2023-02-13 11:53:29,126 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 2023-02-13 11:53:29,131 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 2023-02-13 11:53:29,134 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 2023-02-13 11:53:29,138 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 2023-02-13 11:53:29,142 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e 2023-02-13 11:53:29,149 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a 2023-02-13 11:53:29,149 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(310): Store=5249cf8a85fb696a935f1c38070bb429/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:29,149 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,150 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2 2023-02-13 11:53:29,150 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2 2023-02-13 11:53:29,150 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 5249cf8a85fb696a935f1c38070bb429 columnFamilyName cf2 2023-02-13 11:53:29,160 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 2023-02-13 11:53:29,167 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb 2023-02-13 11:53:29,171 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 2023-02-13 11:53:29,179 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c 2023-02-13 11:53:29,186 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a 2023-02-13 11:53:29,191 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-02-13 11:53:29,191 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 2023-02-13 11:53:29,194 DEBUG [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-02-13 11:53:29,195 DEBUG [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 2023-02-13 11:53:29,196 INFO [StoreOpener-5249cf8a85fb696a935f1c38070bb429-1] regionserver.HStore(310): Store=5249cf8a85fb696a935f1c38070bb429/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:29,196 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 11:53:29,197 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,198 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.splits 2023-02-13 11:53:29,200 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:29,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 5249cf8a85fb696a935f1c38070bb429; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60371476, jitterRate=-0.10039490461349487}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:29,202 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 5249cf8a85fb696a935f1c38070bb429: Running coprocessor pre-open hook at 1676289209099Writing region info on filesystem at 1676289209099Initializing all the Stores at 1676289209100 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289209101 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289209112 (+11 ms)Cleaning up temporary data from old regions at 1676289209198 (+86 ms)Cleaning up detritus from prior splits at 1676289209199 (+1 ms)Running coprocessor post-open hooks at 1676289209202 (+3 ms)Region opened successfully at 1676289209202 2023-02-13 11:53:29,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429., pid=48, masterSystemTime=1676289209094 2023-02-13 11:53:29,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:29,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:29,205 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:29,205 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:29,206 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 11:53:29,206 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1912): 5249cf8a85fb696a935f1c38070bb429/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:29,206 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 11:53:29,207 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 5249cf8a85fb696a935f1c38070bb429/cf2 in testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,207 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1912): 5249cf8a85fb696a935f1c38070bb429/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:29,208 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 5249cf8a85fb696a935f1c38070bb429/cf1 in testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,208 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp, totalSize=34.9 K 2023-02-13 11:53:29,208 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp, totalSize=34.9 K 2023-02-13 11:53:29,208 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 6b4fd7e2853f4617834a0d178456c990, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289201270 2023-02-13 11:53:29,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,208 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting 0f6d684af4f54f40b9c588aa4e8c0413, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289201270 2023-02-13 11:53:29,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,209 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:29,209 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting ce063ca9705c495b87f539ed8b997f3a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289201546 2023-02-13 11:53:29,209 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting c84a48c1b00540939eec2407b845d087, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289201546 2023-02-13 11:53:29,209 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289209209"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289209209"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289209209"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289209209"}]},"ts":"1676289209209"} 2023-02-13 11:53:29,209 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting aff3a737759a4346b37626329f50fca0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289202270 2023-02-13 11:53:29,209 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting d006d5a631e645efbabe9871463bd806, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289202270 2023-02-13 11:53:29,209 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting bfc806e60fe7474db07acb9a62bf9b6e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289202532 2023-02-13 11:53:29,210 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting ace72e50efed4cd1bc51cdd7030c153c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289202532 2023-02-13 11:53:29,210 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 5dfaaebc8604485ebc13056a00bf3b10, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289202772 2023-02-13 11:53:29,210 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting c8208bc7abca4a3fa622fe266751c97a, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289202772 2023-02-13 11:53:29,210 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 54e71050b34e4772895f463aff5b16be, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289203072 2023-02-13 11:53:29,210 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting 3505ceea0e064689a125b46cdf34c4eb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289203072 2023-02-13 11:53:29,211 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting bd4f2bff57d04d3eb83e428e35483a07, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289203334 2023-02-13 11:53:29,211 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting 82b51dfe6c894a07915cbfb80066fb30, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289203334 2023-02-13 11:53:29,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-02-13 11:53:29,215 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 in 269 msec 2023-02-13 11:53:29,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN in 1.6120 sec 2023-02-13 11:53:29,240 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 5249cf8a85fb696a935f1c38070bb429#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:29,244 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] throttle.PressureAwareThroughputController(145): 5249cf8a85fb696a935f1c38070bb429#cf1#compaction#51 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:29,268 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf2/8875086d4013463bb95620e337357ade as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/8875086d4013463bb95620e337357ade 2023-02-13 11:53:29,273 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/.tmp/cf1/811cf3308f9d4dc2bca82a27b47fe456 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/811cf3308f9d4dc2bca82a27b47fe456 2023-02-13 11:53:29,275 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5249cf8a85fb696a935f1c38070bb429/cf2 of 5249cf8a85fb696a935f1c38070bb429 into 8875086d4013463bb95620e337357ade(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:29,275 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:29,275 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429., storeName=5249cf8a85fb696a935f1c38070bb429/cf2, priority=9, startTime=1676289209204; duration=0sec 2023-02-13 11:53:29,276 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:29,279 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 5249cf8a85fb696a935f1c38070bb429/cf1 of 5249cf8a85fb696a935f1c38070bb429 into 811cf3308f9d4dc2bca82a27b47fe456(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:29,279 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 5249cf8a85fb696a935f1c38070bb429: 2023-02-13 11:53:29,279 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429., storeName=5249cf8a85fb696a935f1c38070bb429/cf1, priority=9, startTime=1676289209204; duration=0sec 2023-02-13 11:53:29,279 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:29,403 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-02-13 11:53:29,404 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-02-13 11:53:29,404 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-02-13 11:53:29,427 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:29,429 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-02-13 11:53:29,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRollbackAndDoubleExecution 2023-02-13 11:53:29,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,433 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 11:53:29,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289209433"}]},"ts":"1676289209433"} 2023-02-13 11:53:29,434 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 11:53:29,449 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-02-13 11:53:29,450 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN}] 2023-02-13 11:53:29,450 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN 2023-02-13 11:53:29,451 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:29,451 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289209451"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289209451"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289209451"}]},"ts":"1676289209451"} 2023-02-13 11:53:29,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:29,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 11:53:29,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 5249cf8a85fb696a935f1c38070bb429, disabling compactions & flushes 2023-02-13 11:53:29,605 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. after waiting 0 ms 2023-02-13 11:53:29,605 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,607 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07] to archive 2023-02-13 11:53:29,609 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:29,614 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/6b4fd7e2853f4617834a0d178456c990 2023-02-13 11:53:29,617 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/ce063ca9705c495b87f539ed8b997f3a 2023-02-13 11:53:29,619 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/aff3a737759a4346b37626329f50fca0 2023-02-13 11:53:29,621 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bfc806e60fe7474db07acb9a62bf9b6e 2023-02-13 11:53:29,624 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/5dfaaebc8604485ebc13056a00bf3b10 2023-02-13 11:53:29,626 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/54e71050b34e4772895f463aff5b16be 2023-02-13 11:53:29,628 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/bd4f2bff57d04d3eb83e428e35483a07 2023-02-13 11:53:29,633 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30] to archive 2023-02-13 11:53:29,633 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:29,635 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/0f6d684af4f54f40b9c588aa4e8c0413 2023-02-13 11:53:29,636 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c84a48c1b00540939eec2407b845d087 2023-02-13 11:53:29,637 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/d006d5a631e645efbabe9871463bd806 2023-02-13 11:53:29,639 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/ace72e50efed4cd1bc51cdd7030c153c 2023-02-13 11:53:29,640 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/c8208bc7abca4a3fa622fe266751c97a 2023-02-13 11:53:29,641 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/3505ceea0e064689a125b46cdf34c4eb 2023-02-13 11:53:29,643 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/82b51dfe6c894a07915cbfb80066fb30 2023-02-13 11:53:29,648 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:53:29,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:29,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429. 2023-02-13 11:53:29,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 5249cf8a85fb696a935f1c38070bb429: Waiting for close lock at 1676289209605Running coprocessor pre-close hooks at 1676289209605Disabling compacts and flushes for region at 1676289209605Disabling writes for close at 1676289209605Writing region close event to WAL at 1676289209644 (+39 ms)Running coprocessor post-close hooks at 1676289209649 (+5 ms)Closed at 1676289209649 2023-02-13 11:53:29,651 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,651 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=5249cf8a85fb696a935f1c38070bb429, regionState=CLOSED 2023-02-13 11:53:29,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289209651"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289209651"}]},"ts":"1676289209651"} 2023-02-13 11:53:29,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-02-13 11:53:29,656 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 5249cf8a85fb696a935f1c38070bb429, server=jenkins-hbase16.apache.org,44743,1676289185212 in 201 msec 2023-02-13 11:53:29,658 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-02-13 11:53:29,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, UNASSIGN in 206 msec 2023-02-13 11:53:29,660 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289209659"}]},"ts":"1676289209659"} 2023-02-13 11:53:29,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 11:53:29,668 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-02-13 11:53:29,671 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 240 msec 2023-02-13 11:53:29,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-02-13 11:53:29,737 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-02-13 11:53:29,739 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRollbackAndDoubleExecution 2023-02-13 11:53:29,740 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,742 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,743 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 11:53:29,746 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,749 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits] 2023-02-13 11:53:29,752 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/811cf3308f9d4dc2bca82a27b47fe456 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf1/811cf3308f9d4dc2bca82a27b47fe456 2023-02-13 11:53:29,754 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/8875086d4013463bb95620e337357ade to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/cf2/8875086d4013463bb95620e337357ade 2023-02-13 11:53:29,756 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429/recovered.edits/90.seqid 2023-02-13 11:53:29,757 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackAndDoubleExecution/5249cf8a85fb696a935f1c38070bb429 2023-02-13 11:53:29,757 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-02-13 11:53:29,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,762 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-02-13 11:53:29,763 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-02-13 11:53:29,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-02-13 11:53:29,764 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289209764"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:29,766 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 11:53:29,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 5249cf8a85fb696a935f1c38070bb429, NAME => 'testRollbackAndDoubleExecution,,1676289200761.5249cf8a85fb696a935f1c38070bb429.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 11:53:29,766 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-02-13 11:53:29,766 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289209766"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:29,767 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-02-13 11:53:29,777 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:53:29,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 38 msec 2023-02-13 11:53:29,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-02-13 11:53:29,845 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-02-13 11:53:29,868 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=406 (was 404) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:45426 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38182 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:44792 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38210 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38234 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:55446 [Waiting for operation #52] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37456 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38228 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38198 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase16:44743-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:60462 [Waiting for operation #40] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:60540 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38212 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:45402 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38236 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:59698 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:44768 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 871), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 300) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5224 (was 5408) 2023-02-13 11:53:29,881 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=406, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=169, AvailableMemoryMB=5223 2023-02-13 11:53:29,881 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:29,893 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:29,894 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:29,896 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:29,898 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:29,899 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:29,901 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:29,902 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 empty. 2023-02-13 11:53:29,903 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:29,903 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 11:53:29,919 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:29,920 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0ea26b5dc2d93e295dcd198d6f8d85b4, NAME => 'testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1603): Closing 0ea26b5dc2d93e295dcd198d6f8d85b4, disabling compactions & flushes 2023-02-13 11:53:29,932 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. after waiting 0 ms 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:29,932 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:29,932 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1557): Region close journal for 0ea26b5dc2d93e295dcd198d6f8d85b4: Waiting for close lock at 1676289209932Disabling compacts and flushes for region at 1676289209932Disabling writes for close at 1676289209932Writing region close event to WAL at 1676289209932Closed at 1676289209932 2023-02-13 11:53:29,935 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:29,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289209936"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289209936"}]},"ts":"1676289209936"} 2023-02-13 11:53:29,938 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:29,939 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:29,940 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289209939"}]},"ts":"1676289209939"} 2023-02-13 11:53:29,941 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-02-13 11:53:29,968 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:29,970 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:29,970 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:29,970 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:29,970 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:29,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, ASSIGN}] 2023-02-13 11:53:29,972 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, ASSIGN 2023-02-13 11:53:29,972 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:30,122 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:30,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=0ea26b5dc2d93e295dcd198d6f8d85b4, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289210125"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289210125"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289210125"}]},"ts":"1676289210125"} 2023-02-13 11:53:30,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 0ea26b5dc2d93e295dcd198d6f8d85b4, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:30,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0ea26b5dc2d93e295dcd198d6f8d85b4, NAME => 'testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:30,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:30,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:30,291 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,292 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,293 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,295 DEBUG [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf1 2023-02-13 11:53:30,295 DEBUG [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf1 2023-02-13 11:53:30,296 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ea26b5dc2d93e295dcd198d6f8d85b4 columnFamilyName cf1 2023-02-13 11:53:30,297 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] regionserver.HStore(310): Store=0ea26b5dc2d93e295dcd198d6f8d85b4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,297 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,298 DEBUG [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf2 2023-02-13 11:53:30,298 DEBUG [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf2 2023-02-13 11:53:30,299 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0ea26b5dc2d93e295dcd198d6f8d85b4 columnFamilyName cf2 2023-02-13 11:53:30,299 INFO [StoreOpener-0ea26b5dc2d93e295dcd198d6f8d85b4-1] regionserver.HStore(310): Store=0ea26b5dc2d93e295dcd198d6f8d85b4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:30,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:30,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 0ea26b5dc2d93e295dcd198d6f8d85b4; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64356048, jitterRate=-0.04102015495300293}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:30,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 0ea26b5dc2d93e295dcd198d6f8d85b4: Running coprocessor pre-open hook at 1676289210292Writing region info on filesystem at 1676289210292Initializing all the Stores at 1676289210293 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210293Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210293Cleaning up temporary data from old regions at 1676289210301 (+8 ms)Cleaning up detritus from prior splits at 1676289210301Running coprocessor post-open hooks at 1676289210309 (+8 ms)Region opened successfully at 1676289210309 2023-02-13 11:53:30,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4., pid=55, masterSystemTime=1676289210282 2023-02-13 11:53:30,311 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,311 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=0ea26b5dc2d93e295dcd198d6f8d85b4, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289210311"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289210311"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289210311"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289210311"}]},"ts":"1676289210311"} 2023-02-13 11:53:30,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-02-13 11:53:30,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 0ea26b5dc2d93e295dcd198d6f8d85b4, server=jenkins-hbase16.apache.org,43159,1676289185121 in 184 msec 2023-02-13 11:53:30,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-02-13 11:53:30,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, ASSIGN in 345 msec 2023-02-13 11:53:30,317 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:30,317 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289210317"}]},"ts":"1676289210317"} 2023-02-13 11:53:30,318 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-02-13 11:53:30,327 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:30,329 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 433 msec 2023-02-13 11:53:30,387 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:30,397 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,398 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,399 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,400 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,400 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, UNASSIGN}] 2023-02-13 11:53:30,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, UNASSIGN 2023-02-13 11:53:30,408 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=0ea26b5dc2d93e295dcd198d6f8d85b4, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,408 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289210408"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289210408"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289210408"}]},"ts":"1676289210408"} 2023-02-13 11:53:30,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 0ea26b5dc2d93e295dcd198d6f8d85b4, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:30,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 0ea26b5dc2d93e295dcd198d6f8d85b4, disabling compactions & flushes 2023-02-13 11:53:30,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. after waiting 0 ms 2023-02-13 11:53:30,565 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,574 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 11:53:30,575 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:30,576 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4. 2023-02-13 11:53:30,576 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 0ea26b5dc2d93e295dcd198d6f8d85b4: Waiting for close lock at 1676289210563Running coprocessor pre-close hooks at 1676289210563Disabling compacts and flushes for region at 1676289210563Disabling writes for close at 1676289210565 (+2 ms)Writing region close event to WAL at 1676289210566 (+1 ms)Running coprocessor post-close hooks at 1676289210575 (+9 ms)Closed at 1676289210576 (+1 ms) 2023-02-13 11:53:30,578 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:30,578 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=0ea26b5dc2d93e295dcd198d6f8d85b4, regionState=CLOSED 2023-02-13 11:53:30,578 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289210578"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289210578"}]},"ts":"1676289210578"} 2023-02-13 11:53:30,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-02-13 11:53:30,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 0ea26b5dc2d93e295dcd198d6f8d85b4, server=jenkins-hbase16.apache.org,43159,1676289185121 in 170 msec 2023-02-13 11:53:30,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-02-13 11:53:30,583 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=0ea26b5dc2d93e295dcd198d6f8d85b4, UNASSIGN in 175 msec 2023-02-13 11:53:30,613 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 11:53:30,615 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-02-13 11:53:30,616 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1676289210616"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1676289210616"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1676289210616"}]},"ts":"1676289210616"} 2023-02-13 11:53:30,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289210616"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289210616"}]},"ts":"1676289210616"} 2023-02-13 11:53:30,617 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210616"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289210616"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289210616"}]},"ts":"1676289210616"} 2023-02-13 11:53:30,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, ASSIGN}] 2023-02-13 11:53:30,621 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, ASSIGN 2023-02-13 11:53:30,622 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:30,622 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, ASSIGN 2023-02-13 11:53:30,623 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:30,772 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:53:30,774 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=567a13b6e959216f77208d4ff8cf6917, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,774 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210774"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289210774"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289210774"}]},"ts":"1676289210774"} 2023-02-13 11:53:30,776 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 567a13b6e959216f77208d4ff8cf6917, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:30,778 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=57492564cd0c4ed8e8e634afd1ddfc55, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,778 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210778"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289210778"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289210778"}]},"ts":"1676289210778"} 2023-02-13 11:53:30,780 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 57492564cd0c4ed8e8e634afd1ddfc55, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:30,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:30,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 57492564cd0c4ed8e8e634afd1ddfc55, NAME => 'testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 11:53:30,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:30,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:30,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,933 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,934 DEBUG [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf1 2023-02-13 11:53:30,934 DEBUG [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf1 2023-02-13 11:53:30,935 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57492564cd0c4ed8e8e634afd1ddfc55 columnFamilyName cf1 2023-02-13 11:53:30,935 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] regionserver.HStore(310): Store=57492564cd0c4ed8e8e634afd1ddfc55/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,935 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,936 DEBUG [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf2 2023-02-13 11:53:30,936 DEBUG [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf2 2023-02-13 11:53:30,937 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 57492564cd0c4ed8e8e634afd1ddfc55 columnFamilyName cf2 2023-02-13 11:53:30,937 INFO [StoreOpener-57492564cd0c4ed8e8e634afd1ddfc55-1] regionserver.HStore(310): Store=57492564cd0c4ed8e8e634afd1ddfc55/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,941 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:30,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:30,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 57492564cd0c4ed8e8e634afd1ddfc55; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70357197, jitterRate=0.04840393364429474}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:30,943 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 57492564cd0c4ed8e8e634afd1ddfc55: Running coprocessor pre-open hook at 1676289210932Writing region info on filesystem at 1676289210932Initializing all the Stores at 1676289210933 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210933Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210933Cleaning up temporary data from old regions at 1676289210939 (+6 ms)Cleaning up detritus from prior splits at 1676289210940 (+1 ms)Running coprocessor post-open hooks at 1676289210943 (+3 ms)Region opened successfully at 1676289210943 2023-02-13 11:53:30,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55., pid=62, masterSystemTime=1676289210927 2023-02-13 11:53:30,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:30,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:30,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:30,946 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 567a13b6e959216f77208d4ff8cf6917, NAME => 'testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 11:53:30,946 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=57492564cd0c4ed8e8e634afd1ddfc55, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:30,947 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210946"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289210946"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289210946"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289210946"}]},"ts":"1676289210946"} 2023-02-13 11:53:30,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:30,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,947 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,949 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,951 DEBUG [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf1 2023-02-13 11:53:30,951 DEBUG [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf1 2023-02-13 11:53:30,952 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 567a13b6e959216f77208d4ff8cf6917 columnFamilyName cf1 2023-02-13 11:53:30,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-02-13 11:53:30,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 57492564cd0c4ed8e8e634afd1ddfc55, server=jenkins-hbase16.apache.org,43159,1676289185121 in 169 msec 2023-02-13 11:53:30,952 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] regionserver.HStore(310): Store=567a13b6e959216f77208d4ff8cf6917/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,952 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,954 DEBUG [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf2 2023-02-13 11:53:30,954 DEBUG [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf2 2023-02-13 11:53:30,954 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 567a13b6e959216f77208d4ff8cf6917 columnFamilyName cf2 2023-02-13 11:53:30,955 INFO [StoreOpener-567a13b6e959216f77208d4ff8cf6917-1] regionserver.HStore(310): Store=567a13b6e959216f77208d4ff8cf6917/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:30,956 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,957 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, ASSIGN in 331 msec 2023-02-13 11:53:30,959 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,961 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:30,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:30,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 567a13b6e959216f77208d4ff8cf6917; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72497164, jitterRate=0.08029192686080933}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:30,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 567a13b6e959216f77208d4ff8cf6917: Running coprocessor pre-open hook at 1676289210948Writing region info on filesystem at 1676289210948Initializing all the Stores at 1676289210949 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210949Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289210949Cleaning up temporary data from old regions at 1676289210960 (+11 ms)Cleaning up detritus from prior splits at 1676289210960Running coprocessor post-open hooks at 1676289210964 (+4 ms)Region opened successfully at 1676289210964 2023-02-13 11:53:30,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917., pid=61, masterSystemTime=1676289210927 2023-02-13 11:53:30,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:30,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:30,966 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=567a13b6e959216f77208d4ff8cf6917, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:30,967 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289210966"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289210966"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289210966"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289210966"}]},"ts":"1676289210966"} 2023-02-13 11:53:30,969 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-02-13 11:53:30,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 567a13b6e959216f77208d4ff8cf6917, server=jenkins-hbase16.apache.org,43159,1676289185121 in 192 msec 2023-02-13 11:53:30,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-02-13 11:53:30,971 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, ASSIGN in 349 msec 2023-02-13 11:53:30,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 in 574 msec 2023-02-13 11:53:31,153 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:53:31,157 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:31,158 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,169 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 11:53:31,170 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289211170"}]},"ts":"1676289211170"} 2023-02-13 11:53:31,171 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-02-13 11:53:31,199 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-02-13 11:53:31,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, UNASSIGN}] 2023-02-13 11:53:31,201 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, UNASSIGN 2023-02-13 11:53:31,201 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=57492564cd0c4ed8e8e634afd1ddfc55, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:31,202 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289211201"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289211201"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289211201"}]},"ts":"1676289211201"} 2023-02-13 11:53:31,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 57492564cd0c4ed8e8e634afd1ddfc55, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:31,205 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, UNASSIGN 2023-02-13 11:53:31,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=567a13b6e959216f77208d4ff8cf6917, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:31,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289211206"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289211206"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289211206"}]},"ts":"1676289211206"} 2023-02-13 11:53:31,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 567a13b6e959216f77208d4ff8cf6917, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:31,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 11:53:31,355 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:31,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 57492564cd0c4ed8e8e634afd1ddfc55, disabling compactions & flushes 2023-02-13 11:53:31,357 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:31,357 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:31,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. after waiting 0 ms 2023-02-13 11:53:31,358 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:31,366 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 11:53:31,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:31,367 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55. 2023-02-13 11:53:31,367 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 57492564cd0c4ed8e8e634afd1ddfc55: Waiting for close lock at 1676289211356Running coprocessor pre-close hooks at 1676289211356Disabling compacts and flushes for region at 1676289211356Disabling writes for close at 1676289211358 (+2 ms)Writing region close event to WAL at 1676289211358Running coprocessor post-close hooks at 1676289211367 (+9 ms)Closed at 1676289211367 2023-02-13 11:53:31,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:31,369 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:31,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 567a13b6e959216f77208d4ff8cf6917, disabling compactions & flushes 2023-02-13 11:53:31,371 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:31,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:31,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. after waiting 0 ms 2023-02-13 11:53:31,371 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:31,371 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=57492564cd0c4ed8e8e634afd1ddfc55, regionState=CLOSED 2023-02-13 11:53:31,371 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289211371"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289211371"}]},"ts":"1676289211371"} 2023-02-13 11:53:31,375 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 11:53:31,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-02-13 11:53:31,376 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 57492564cd0c4ed8e8e634afd1ddfc55, server=jenkins-hbase16.apache.org,43159,1676289185121 in 170 msec 2023-02-13 11:53:31,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:31,376 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917. 2023-02-13 11:53:31,376 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 567a13b6e959216f77208d4ff8cf6917: Waiting for close lock at 1676289211370Running coprocessor pre-close hooks at 1676289211370Disabling compacts and flushes for region at 1676289211370Disabling writes for close at 1676289211371 (+1 ms)Writing region close event to WAL at 1676289211371Running coprocessor post-close hooks at 1676289211376 (+5 ms)Closed at 1676289211376 2023-02-13 11:53:31,378 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=57492564cd0c4ed8e8e634afd1ddfc55, UNASSIGN in 176 msec 2023-02-13 11:53:31,378 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:31,378 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=567a13b6e959216f77208d4ff8cf6917, regionState=CLOSED 2023-02-13 11:53:31,379 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289211378"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289211378"}]},"ts":"1676289211378"} 2023-02-13 11:53:31,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-02-13 11:53:31,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 567a13b6e959216f77208d4ff8cf6917, server=jenkins-hbase16.apache.org,43159,1676289185121 in 171 msec 2023-02-13 11:53:31,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-02-13 11:53:31,382 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=567a13b6e959216f77208d4ff8cf6917, UNASSIGN in 181 msec 2023-02-13 11:53:31,383 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289211383"}]},"ts":"1676289211383"} 2023-02-13 11:53:31,384 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-02-13 11:53:31,416 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-02-13 11:53:31,419 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 258 msec 2023-02-13 11:53:31,472 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-02-13 11:53:31,472 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-02-13 11:53:31,473 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,474 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,475 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,476 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 11:53:31,481 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:31,481 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:31,481 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:31,484 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/recovered.edits] 2023-02-13 11:53:31,484 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/recovered.edits] 2023-02-13 11:53:31,484 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/recovered.edits] 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/recovered.edits/4.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4/recovered.edits/4.seqid 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/recovered.edits/7.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55/recovered.edits/7.seqid 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/recovered.edits/7.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917/recovered.edits/7.seqid 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/0ea26b5dc2d93e295dcd198d6f8d85b4 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:53:31,489 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionNoStoreFile/567a13b6e959216f77208d4ff8cf6917 2023-02-13 11:53:31,490 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-02-13 11:53:31,491 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,493 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-02-13 11:53:31,495 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-02-13 11:53:31,496 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,496 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-02-13 11:53:31,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289211496"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:31,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289211496"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:31,496 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289211496"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:31,497 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:53:31,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0ea26b5dc2d93e295dcd198d6f8d85b4, NAME => 'testSplitTableRegionNoStoreFile,,1676289209894.0ea26b5dc2d93e295dcd198d6f8d85b4.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 567a13b6e959216f77208d4ff8cf6917, NAME => 'testSplitTableRegionNoStoreFile,,1676289210397.567a13b6e959216f77208d4ff8cf6917.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 57492564cd0c4ed8e8e634afd1ddfc55, NAME => 'testSplitTableRegionNoStoreFile,41,1676289210397.57492564cd0c4ed8e8e634afd1ddfc55.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 11:53:31,497 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-02-13 11:53:31,497 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289211497"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:31,499 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-02-13 11:53:31,529 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:53:31,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 56 msec 2023-02-13 11:53:31,577 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-02-13 11:53:31,578 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-02-13 11:53:31,590 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=410 (was 406) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_520044186_15 at /127.0.0.1:38198 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_520044186_15 at /127.0.0.1:60462 [Waiting for operation #41] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=855 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 308), ProcessCount=169 (was 169), AvailableMemoryMB=5229 (was 5223) - AvailableMemoryMB LEAK? - 2023-02-13 11:53:31,603 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=410, OpenFileDescriptor=855, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=169, AvailableMemoryMB=5227 2023-02-13 11:53:31,604 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:31,649 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:31,650 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:31,652 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:31,655 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:31,656 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:31,659 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:31,660 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 empty. 2023-02-13 11:53:31,661 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:31,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 11:53:31,670 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:31,672 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 1f69a2ae9ee6751a9ecdbbcfc72870d6, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1603): Closing 1f69a2ae9ee6751a9ecdbbcfc72870d6, disabling compactions & flushes 2023-02-13 11:53:31,682 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. after waiting 0 ms 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:31,682 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:31,682 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: Waiting for close lock at 1676289211682Disabling compacts and flushes for region at 1676289211682Disabling writes for close at 1676289211682Writing region close event to WAL at 1676289211682Closed at 1676289211682 2023-02-13 11:53:31,684 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:31,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289211685"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289211685"}]},"ts":"1676289211685"} 2023-02-13 11:53:31,686 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:31,687 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:31,687 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289211687"}]},"ts":"1676289211687"} 2023-02-13 11:53:31,688 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-02-13 11:53:31,766 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:31,768 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:31,768 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:31,768 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:31,768 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:31,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, ASSIGN}] 2023-02-13 11:53:31,771 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, ASSIGN 2023-02-13 11:53:31,772 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:53:31,922 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:31,923 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=1f69a2ae9ee6751a9ecdbbcfc72870d6, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:31,923 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289211923"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289211923"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289211923"}]},"ts":"1676289211923"} 2023-02-13 11:53:31,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure 1f69a2ae9ee6751a9ecdbbcfc72870d6, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:32,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 1f69a2ae9ee6751a9ecdbbcfc72870d6, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:32,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:32,081 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,082 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,084 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,086 DEBUG [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1 2023-02-13 11:53:32,086 DEBUG [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1 2023-02-13 11:53:32,087 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f69a2ae9ee6751a9ecdbbcfc72870d6 columnFamilyName cf1 2023-02-13 11:53:32,087 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] regionserver.HStore(310): Store=1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:32,087 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,090 DEBUG [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2 2023-02-13 11:53:32,090 DEBUG [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2 2023-02-13 11:53:32,090 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1f69a2ae9ee6751a9ecdbbcfc72870d6 columnFamilyName cf2 2023-02-13 11:53:32,091 INFO [StoreOpener-1f69a2ae9ee6751a9ecdbbcfc72870d6-1] regionserver.HStore(310): Store=1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:32,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:32,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:32,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:32,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 1f69a2ae9ee6751a9ecdbbcfc72870d6; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69848431, jitterRate=0.04082272946834564}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:32,100 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: Running coprocessor pre-open hook at 1676289212082Writing region info on filesystem at 1676289212082Initializing all the Stores at 1676289212083 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289212083Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289212083Cleaning up temporary data from old regions at 1676289212094 (+11 ms)Cleaning up detritus from prior splits at 1676289212094Running coprocessor post-open hooks at 1676289212100 (+6 ms)Region opened successfully at 1676289212100 2023-02-13 11:53:32,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6., pid=71, masterSystemTime=1676289212076 2023-02-13 11:53:32,102 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=1f69a2ae9ee6751a9ecdbbcfc72870d6, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,103 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289212102"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289212102"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289212102"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289212102"}]},"ts":"1676289212102"} 2023-02-13 11:53:32,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-02-13 11:53:32,106 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure 1f69a2ae9ee6751a9ecdbbcfc72870d6, server=jenkins-hbase16.apache.org,44743,1676289185212 in 179 msec 2023-02-13 11:53:32,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-02-13 11:53:32,107 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, ASSIGN in 338 msec 2023-02-13 11:53:32,108 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:32,108 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289212108"}]},"ts":"1676289212108"} 2023-02-13 11:53:32,109 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-02-13 11:53:32,129 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:32,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 478 msec 2023-02-13 11:53:32,160 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:32,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,161 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,161 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,161 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:32,161 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:32,162 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,162 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,168 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,168 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,168 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,169 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,169 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,169 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,169 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,170 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:32,170 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:32,170 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,171 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:32,172 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,177 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,215 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/aa6a7f55aa5a4567815c088470a1e34c 2023-02-13 11:53:32,246 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/aa3c021f3ab4409a920f31854688350c 2023-02-13 11:53:32,251 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/aa6a7f55aa5a4567815c088470a1e34c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c 2023-02-13 11:53:32,256 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:32,257 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/aa3c021f3ab4409a920f31854688350c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c 2023-02-13 11:53:32,263 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:53:32,267 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 95ms, sequenceid=6, compaction requested=false 2023-02-13 11:53:32,267 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,267 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:32,268 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,268 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:32,268 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:32,268 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,268 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,268 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,438 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,438 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,438 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,438 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:32,440 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@415f607e[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:32,440 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:32,440 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,519 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,519 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,519 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,520 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:32,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,520 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:32,535 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,535 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,535 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,549 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,549 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,552 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:32,552 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:32,552 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:32,552 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:32,552 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:32,552 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,552 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:32,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,565 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,566 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,566 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,567 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,567 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,567 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,568 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,568 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,569 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,607 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,607 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,608 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:32,607 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,607 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,608 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,608 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:32,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,609 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,609 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,609 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,609 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,709 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:32,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:32,726 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:32,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:32,727 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,727 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:32,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:32,728 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,728 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,735 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,735 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,735 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:32,735 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,736 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:32,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:32,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,741 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:32,741 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:32,741 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:32,741 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,741 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,741 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:32,749 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,755 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/72b2f4bd891e400ca142034de90354de 2023-02-13 11:53:32,757 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,757 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,757 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,757 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,757 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,772 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/a818e862481a4880be8cf5a731ea9d2c 2023-02-13 11:53:32,776 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/72b2f4bd891e400ca142034de90354de as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de 2023-02-13 11:53:32,781 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:32,782 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/a818e862481a4880be8cf5a731ea9d2c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c 2023-02-13 11:53:32,787 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:53:32,787 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 46ms, sequenceid=19, compaction requested=false 2023-02-13 11:53:32,788 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:32,788 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:32,788 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:32,788 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:32,788 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,788 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:32,788 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:32,799 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,799 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,799 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,799 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:32,799 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,800 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:32,801 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7690189a[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:32,801 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:32,802 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,816 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:32,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:32,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:32,816 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,827 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,835 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:32,835 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,835 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,835 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:32,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:32,839 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:32,839 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:32,839 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:32,839 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:32,839 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,839 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:32,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:32,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,850 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,851 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,851 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,852 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:32,853 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,854 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,854 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,866 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:32,866 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:32,866 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:32,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,867 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:32,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,867 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,967 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:32,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:32,983 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:32,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:32,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,985 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:32,985 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:32,985 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:32,985 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:32,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:32,999 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,999 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,999 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:32,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:32,999 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:32,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:32,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,000 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,001 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,001 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,001 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,001 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,001 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,001 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,001 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,001 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,004 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,004 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:33,004 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:33,004 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,004 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:33,004 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:33,022 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/88a60af79f034c6ba19297ff86ae6c44 2023-02-13 11:53:33,042 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/cb89f41aa7c94824a9b0c33301ed6394 2023-02-13 11:53:33,047 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,047 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,048 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,049 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/88a60af79f034c6ba19297ff86ae6c44 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44 2023-02-13 11:53:33,054 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:33,055 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/cb89f41aa7c94824a9b0c33301ed6394 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394 2023-02-13 11:53:33,061 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,062 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,062 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,063 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:53:33,064 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 60ms, sequenceid=32, compaction requested=true 2023-02-13 11:53:33,064 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:33,064 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,064 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:33,065 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:33,065 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,065 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,065 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,077 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,077 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,077 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,077 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:33,079 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@472f5e5a[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:33,079 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:33,079 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,091 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,091 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:33,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:33,091 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,091 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,092 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:33,092 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,092 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,092 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:33,107 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,107 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,107 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,118 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,118 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,118 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,119 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,132 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,132 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:33,135 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:33,135 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:33,135 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:33,135 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:33,135 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,136 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,143 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,144 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,143 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,144 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,145 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,145 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,145 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,145 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,147 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,147 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,148 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,157 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:33,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,158 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,159 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,159 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,159 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,159 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,258 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:33,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:33,274 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:33,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:33,275 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,275 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:33,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:33,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,307 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,307 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,308 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,308 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:33,307 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,308 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,309 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,309 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:33,310 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:33,310 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,310 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:33,310 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:33,318 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,321 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/6db1be2e94604541bc57013dd4355d72 2023-02-13 11:53:33,327 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,327 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,327 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,327 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,327 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,339 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/e8f667ea97404b249e323a8f4f73f632 2023-02-13 11:53:33,345 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/6db1be2e94604541bc57013dd4355d72 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72 2023-02-13 11:53:33,349 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:33,351 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/e8f667ea97404b249e323a8f4f73f632 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632 2023-02-13 11:53:33,356 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:53:33,357 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 47ms, sequenceid=45, compaction requested=true 2023-02-13 11:53:33,357 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:33,357 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:33,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:33,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,358 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,358 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,368 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,368 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,369 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,369 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:33,370 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@58f66704[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:33,370 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:33,370 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,377 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,377 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,377 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:33,385 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,385 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,385 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,393 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,393 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,393 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,433 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,433 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,433 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,433 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,434 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:33,436 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:33,436 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:33,436 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:33,436 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:33,436 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,436 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,450 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,450 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,450 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,450 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,452 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,452 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,452 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,460 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:33,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,460 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,460 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:33,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,461 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,561 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:33,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:33,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:33,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:33,580 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,580 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:33,580 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:33,581 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,581 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,591 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,591 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,591 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,591 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,591 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,592 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,592 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,592 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:33,593 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:33,593 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,593 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:33,593 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:33,602 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,608 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/5a745d2ca0194394bd9065d889694d16 2023-02-13 11:53:33,611 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,611 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,611 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,612 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,613 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,635 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 2023-02-13 11:53:33,641 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/5a745d2ca0194394bd9065d889694d16 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16 2023-02-13 11:53:33,646 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:33,646 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 2023-02-13 11:53:33,652 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:53:33,653 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 60ms, sequenceid=58, compaction requested=true 2023-02-13 11:53:33,653 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:33,653 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,653 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:33,653 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:33,653 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,653 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,653 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,660 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,660 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,660 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,660 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:33,671 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6290b0f4[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:33,671 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:33,672 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,682 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,682 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,682 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,683 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,683 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,683 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:33,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,699 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,707 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,708 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,708 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:33,711 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:33,711 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:33,711 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:33,711 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:33,711 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,711 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,718 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,718 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,719 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,719 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,719 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,720 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,719 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,720 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,720 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,721 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,721 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,721 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,722 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,752 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,751 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,751 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:33,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,753 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,853 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:33,855 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:33,869 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:33,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:33,870 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,870 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,870 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,870 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:33,870 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:33,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,877 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,877 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,877 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,877 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,878 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,878 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,878 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:33,879 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,879 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:33,879 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,879 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:53:33,885 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,893 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,893 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,893 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,894 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,895 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/b550ed5cd8774bfd9415da720486c212 2023-02-13 11:53:33,910 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/3f665a7c5f2f43a2b5fda71f86783863 2023-02-13 11:53:33,915 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/b550ed5cd8774bfd9415da720486c212 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212 2023-02-13 11:53:33,919 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:33,920 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/3f665a7c5f2f43a2b5fda71f86783863 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863 2023-02-13 11:53:33,925 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:53:33,926 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 47ms, sequenceid=71, compaction requested=true 2023-02-13 11:53:33,926 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:33,926 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:33,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:33,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:33,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:33,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:33,935 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,935 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,935 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,935 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:33,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:33,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@143230f4[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:33,937 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:33,937 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:33,943 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:33,943 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:33,944 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:33,952 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,952 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,952 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,952 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,960 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,960 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,960 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,968 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:33,968 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,969 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:33,973 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:33,973 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:33,973 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:33,973 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:33,973 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,973 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,982 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:33,983 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,983 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:33,983 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,983 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,984 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,985 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:33,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,986 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,986 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,987 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,994 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:33,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:33,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:33,994 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:33,995 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:33,994 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:33,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:33,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:33,995 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:33,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:33,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:33,995 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:33,996 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,096 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:34,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:34,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:34,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:34,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,118 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:34,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:34,119 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,119 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,127 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,127 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,127 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,128 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,128 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,128 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,129 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,129 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,129 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,130 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,130 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,130 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:34,130 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,130 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,131 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,131 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:34,131 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,131 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-02-13 11:53:34,141 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,147 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:34,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,151 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:34,163 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:34,167 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:34,168 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:34,173 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:34,173 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 11:53:34,174 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/dd152a59cc4b456d99d3287f9c93b3fa as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:34,179 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:34,179 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa, entries=9, sequenceid=83, filesize=5.1 K 2023-02-13 11:53:34,180 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 49ms, sequenceid=83, compaction requested=true 2023-02-13 11:53:34,180 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:34,180 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,180 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:34,180 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:34,180 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,180 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,180 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,191 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,191 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,192 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,192 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:34,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29d261a6[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:34,194 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:34,194 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,207 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,207 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,207 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,207 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,208 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:34,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:34,208 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:34,245 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,245 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,245 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,245 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,258 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,257 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,258 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,258 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,258 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:34,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:34,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:34,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:34,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:34,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,274 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,274 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,274 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,275 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,275 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,276 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,277 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,277 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,285 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,285 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(398): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Unable to get data of znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter because node does not exist (not an error) 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,285 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,285 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,286 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,286 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:34,286 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,387 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:34,387 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:34,396 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:34,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:34,398 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,398 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,398 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,398 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:34,398 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:34,399 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,399 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,407 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,407 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,407 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,408 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,408 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,408 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,409 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,410 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:34,410 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,410 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,410 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,410 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,418 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,421 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:34,427 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,427 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,427 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,427 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,428 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,428 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:34,448 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:34,452 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:34,453 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/f1494265d50d4f1486bd7a80899f9453 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:34,457 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:34,457 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 11:53:34,458 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/24036b97aa0442c29cd6bed07023dae9 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:34,463 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:34,463 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9, entries=5, sequenceid=91, filesize=5.0 K 2023-02-13 11:53:34,464 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 53ms, sequenceid=91, compaction requested=true 2023-02-13 11:53:34,464 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:34,464 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,464 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:34,464 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:34,464 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,464 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,464 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,474 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,474 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,474 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,475 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:34,477 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4277eddf[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:34,477 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:34,477 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,485 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,485 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,485 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,485 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,485 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:34,485 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,485 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:34,486 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:34,487 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,487 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,487 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,487 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,487 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:34,499 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,499 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,499 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,499 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,508 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:34,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:34,527 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:34,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:34,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:34,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,527 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,541 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,541 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,542 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,542 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,542 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,542 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,542 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,543 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,544 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,544 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,544 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,544 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,571 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,571 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:34,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,571 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:34,571 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,572 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,572 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,573 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,673 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:34,674 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:34,689 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:53:34,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:53:34,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,691 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,691 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,691 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-02-13 11:53:34,691 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-02-13 11:53:34,692 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,692 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,702 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,702 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:53:34,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,704 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,705 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,706 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,706 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,707 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,707 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,707 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,707 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:53:34,708 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:53:34,709 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,709 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. started... 2023-02-13 11:53:34,709 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-02-13 11:53:34,718 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,721 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:34,725 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:34,727 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,727 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,727 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,727 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,727 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,735 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:34,739 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:34,740 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/eea0bc4826424a578112120bb237b10d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:34,743 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:34,743 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 11:53:34,744 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/0e468b5ee6854258afa1a318fa16684d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:34,749 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:34,749 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d, entries=5, sequenceid=99, filesize=5.0 K 2023-02-13 11:53:34,750 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 41ms, sequenceid=99, compaction requested=true 2023-02-13 11:53:34,750 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: 2023-02-13 11:53:34,750 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:34,750 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:53:34,750 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:53:34,750 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,750 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-02-13 11:53:34,751 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-02-13 11:53:34,760 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,760 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,761 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,761 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:53:34,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-02-13 11:53:34,765 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2507e275[Count = 0] remaining members to acquire global barrier 2023-02-13 11:53:34,765 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-02-13 11:53:34,765 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,775 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,775 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:53:34,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:53:34,775 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:53:34,813 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,813 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-02-13 11:53:34,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,827 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-02-13 11:53:34,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,834 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:53:34,838 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:53:34,838 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-02-13 11:53:34,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:53:34,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:53:34,838 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,838 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,849 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:53:34,849 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,849 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,850 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,850 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:53:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,852 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:53:34,852 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,853 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:53:34,861 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:53:34,860 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-02-13 11:53:34,861 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:53:34,860 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:53:34,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,860 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,861 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:53:34,861 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:53:34,861 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,861 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:53:34,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:53:34,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,862 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:34,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:53:34,961 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:53:34,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:53:34,965 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,966 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:53:34,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:53:34,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:53:34,968 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:53:34,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, UNASSIGN}] 2023-02-13 11:53:34,971 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, UNASSIGN 2023-02-13 11:53:34,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=1f69a2ae9ee6751a9ecdbbcfc72870d6, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:34,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289214971"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289214971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289214971"}]},"ts":"1676289214971"} 2023-02-13 11:53:34,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure 1f69a2ae9ee6751a9ecdbbcfc72870d6, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:35,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 1f69a2ae9ee6751a9ecdbbcfc72870d6, disabling compactions & flushes 2023-02-13 11:53:35,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:35,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:35,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. after waiting 0 ms 2023-02-13 11:53:35,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:35,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6 2/2 column families, dataSize=50 B heapSize=720 B 2023-02-13 11:53:35,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,148 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,158 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,162 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,163 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf1/83f5256fe6254166952803065fef9938 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,167 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,168 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 11:53:35,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.tmp/cf2/1910d209e53546a58c54e4e157f192d5 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,173 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5, entries=1, sequenceid=103, filesize=4.9 K 2023-02-13 11:53:35,175 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 49ms, sequenceid=103, compaction requested=true 2023-02-13 11:53:35,179 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-02-13 11:53:35,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:35,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. 2023-02-13 11:53:35,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 1f69a2ae9ee6751a9ecdbbcfc72870d6: Waiting for close lock at 1676289215126Running coprocessor pre-close hooks at 1676289215126Disabling compacts and flushes for region at 1676289215126Disabling writes for close at 1676289215126Obtaining lock to block concurrent updates at 1676289215126Preparing flush snapshotting stores in 1f69a2ae9ee6751a9ecdbbcfc72870d6 at 1676289215126Finished memstore snapshotting testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6., syncing WAL and waiting on mvcc, flushsize=dataSize=50, getHeapSize=688, getOffHeapSize=0, getCellsCount=2 at 1676289215127 (+1 ms)Flushing stores of testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6. at 1676289215128 (+1 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1: creating writer at 1676289215130 (+2 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1: appending metadata at 1676289215134 (+4 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1: closing flushed file at 1676289215134Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2: creating writer at 1676289215149 (+15 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2: appending metadata at 1676289215150 (+1 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2: closing flushed file at 1676289215150Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1: reopening flushed file at 1676289215164 (+14 ms)Flushing 1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2: reopening flushed file at 1676289215169 (+5 ms)Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for 1f69a2ae9ee6751a9ecdbbcfc72870d6 in 49ms, sequenceid=103, compaction requested=true at 1676289215175 (+6 ms)Writing region close event to WAL at 1676289215177 (+2 ms)Running coprocessor post-close hooks at 1676289215180 (+3 ms)Closed at 1676289215180 2023-02-13 11:53:35,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,182 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=1f69a2ae9ee6751a9ecdbbcfc72870d6, regionState=CLOSED 2023-02-13 11:53:35,182 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289215182"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289215182"}]},"ts":"1676289215182"} 2023-02-13 11:53:35,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-02-13 11:53:35,185 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure 1f69a2ae9ee6751a9ecdbbcfc72870d6, server=jenkins-hbase16.apache.org,44743,1676289185212 in 210 msec 2023-02-13 11:53:35,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-02-13 11:53:35,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, UNASSIGN in 215 msec 2023-02-13 11:53:35,209 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=1f69a2ae9ee6751a9ecdbbcfc72870d6, threads=16 2023-02-13 11:53:35,210 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,210 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,210 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,210 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,212 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,212 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,212 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,212 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,212 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,213 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,219 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:35,219 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863, top=false 2023-02-13 11:53:35,219 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017, top=false 2023-02-13 11:53:35,219 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72, top=false 2023-02-13 11:53:35,220 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:35,221 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:35,226 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632, top=false 2023-02-13 11:53:35,227 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,229 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16, top=false 2023-02-13 11:53:35,229 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:35,230 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,232 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:35,233 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d, top=true 2023-02-13 11:53:35,253 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:35,253 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5, top=true 2023-02-13 11:53:35,256 INFO [StoreFileSplitter-pool-2] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:35,256 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83f5256fe6254166952803065fef9938 2023-02-13 11:53:35,256 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938, top=true 2023-02-13 11:53:35,256 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9, top=true 2023-02-13 11:53:35,295 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,303 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:35,303 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,303 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,312 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,313 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,313 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,313 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,313 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,315 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:35,323 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938 for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,323 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,324 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9 for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,324 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,326 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5 for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,326 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,332 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,333 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212, top=false 2023-02-13 11:53:35,335 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:35,336 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,337 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,337 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:35,339 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:35,339 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d, top=true 2023-02-13 11:53:35,341 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:35,341 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453, top=true 2023-02-13 11:53:35,383 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,383 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,384 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212 for child: 34ef7c7eb6373dbebfbccd213918b582, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,392 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,393 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/.splits/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453 for child: 29381611e213b47c8119c7022001ff40, parent: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,393 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,393 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,415 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,416 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,416 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,417 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,422 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,422 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,770 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394 for region: 1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:35,770 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region 1f69a2ae9ee6751a9ecdbbcfc72870d6 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:53:35,799 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:35,854 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 11:53:35,856 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-02-13 11:53:35,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1676289215858"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1676289215858"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1676289215858"}]},"ts":"1676289215858"} 2023-02-13 11:53:35,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289215858"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289215858"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289215858"}]},"ts":"1676289215858"} 2023-02-13 11:53:35,858 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289215858"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289215858"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289215858"}]},"ts":"1676289215858"} 2023-02-13 11:53:35,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, ASSIGN}] 2023-02-13 11:53:35,862 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, ASSIGN 2023-02-13 11:53:35,863 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:53:35,864 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, ASSIGN 2023-02-13 11:53:35,865 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:53:36,013 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:53:36,015 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=34ef7c7eb6373dbebfbccd213918b582, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:36,015 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289216015"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289216015"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289216015"}]},"ts":"1676289216015"} 2023-02-13 11:53:36,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 34ef7c7eb6373dbebfbccd213918b582, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:36,019 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=29381611e213b47c8119c7022001ff40, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:36,019 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289216019"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289216019"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289216019"}]},"ts":"1676289216019"} 2023-02-13 11:53:36,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure 29381611e213b47c8119c7022001ff40, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:36,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,171 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 29381611e213b47c8119c7022001ff40, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.', STARTKEY => '60', ENDKEY => ''} 2023-02-13 11:53:36,172 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:36,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:36,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,172 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,174 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,175 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1 2023-02-13 11:53:36,175 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1 2023-02-13 11:53:36,176 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29381611e213b47c8119c7022001ff40 columnFamilyName cf1 2023-02-13 11:53:36,196 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-top 2023-02-13 11:53:36,201 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-top 2023-02-13 11:53:36,204 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:36,205 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-top 2023-02-13 11:53:36,210 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-top 2023-02-13 11:53:36,214 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 83f5256fe6254166952803065fef9938 2023-02-13 11:53:36,214 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938 2023-02-13 11:53:36,218 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:36,218 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:36,223 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:36,223 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:36,223 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(310): Store=29381611e213b47c8119c7022001ff40/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:36,224 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,225 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2 2023-02-13 11:53:36,225 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2 2023-02-13 11:53:36,226 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29381611e213b47c8119c7022001ff40 columnFamilyName cf2 2023-02-13 11:53:36,239 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-top 2023-02-13 11:53:36,242 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-top 2023-02-13 11:53:36,247 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-top 2023-02-13 11:53:36,250 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:36,251 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-top 2023-02-13 11:53:36,254 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:36,254 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:36,257 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:36,257 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:36,260 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:36,261 DEBUG [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:36,261 INFO [StoreOpener-29381611e213b47c8119c7022001ff40-1] regionserver.HStore(310): Store=29381611e213b47c8119c7022001ff40/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:36,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:36,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:36,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 29381611e213b47c8119c7022001ff40; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68659206, jitterRate=0.023101896047592163}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:36,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 29381611e213b47c8119c7022001ff40: Running coprocessor pre-open hook at 1676289216172Writing region info on filesystem at 1676289216172Initializing all the Stores at 1676289216173 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289216174 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289216174Cleaning up temporary data from old regions at 1676289216263 (+89 ms)Cleaning up detritus from prior splits at 1676289216263Running coprocessor post-open hooks at 1676289216265 (+2 ms)Region opened successfully at 1676289216265 2023-02-13 11:53:36,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40., pid=78, masterSystemTime=1676289216168 2023-02-13 11:53:36,266 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,267 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:36,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:36,267 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:36,267 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,267 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1912): 29381611e213b47c8119c7022001ff40/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:36,267 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,267 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 29381611e213b47c8119c7022001ff40/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,267 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1912): 29381611e213b47c8119c7022001ff40/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:36,268 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 29381611e213b47c8119c7022001ff40/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,268 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp, totalSize=34.8 K 2023-02-13 11:53:36,268 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp, totalSize=34.8 K 2023-02-13 11:53:36,269 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289212155 2023-02-13 11:53:36,269 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289212155 2023-02-13 11:53:36,269 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=29381611e213b47c8119c7022001ff40, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:36,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289216269"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289216269"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289216269"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289216269"}]},"ts":"1676289216269"} 2023-02-13 11:53:36,270 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289212710 2023-02-13 11:53:36,270 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289212710 2023-02-13 11:53:36,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 34ef7c7eb6373dbebfbccd213918b582, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.', STARTKEY => '', ENDKEY => '60'} 2023-02-13 11:53:36,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:36,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:36,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,273 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289212969 2023-02-13 11:53:36,273 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289212969 2023-02-13 11:53:36,273 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676289214100 2023-02-13 11:53:36,274 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,274 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1676289214100 2023-02-13 11:53:36,274 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,274 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,274 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,275 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1 2023-02-13 11:53:36,275 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1 2023-02-13 11:53:36,275 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,275 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,275 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34ef7c7eb6373dbebfbccd213918b582 columnFamilyName cf1 2023-02-13 11:53:36,275 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-02-13 11:53:36,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure 29381611e213b47c8119c7022001ff40, server=jenkins-hbase16.apache.org,44743,1676289185212 in 251 msec 2023-02-13 11:53:36,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, ASSIGN in 416 msec 2023-02-13 11:53:36,289 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-bottom 2023-02-13 11:53:36,300 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 29381611e213b47c8119c7022001ff40#cf2#compaction#73 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,300 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] throttle.PressureAwareThroughputController(145): 29381611e213b47c8119c7022001ff40#cf1#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,300 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-bottom 2023-02-13 11:53:36,306 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:36,306 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-bottom 2023-02-13 11:53:36,310 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-bottom 2023-02-13 11:53:36,318 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16 2023-02-13 11:53:36,321 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72 2023-02-13 11:53:36,324 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212 2023-02-13 11:53:36,332 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(310): Store=34ef7c7eb6373dbebfbccd213918b582/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:36,332 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,333 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2 2023-02-13 11:53:36,333 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2 2023-02-13 11:53:36,333 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 34ef7c7eb6373dbebfbccd213918b582 columnFamilyName cf2 2023-02-13 11:53:36,336 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 00418fb274c5493684afe78e82550d77: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,337 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp/cf1/00418fb274c5493684afe78e82550d77 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/00418fb274c5493684afe78e82550d77 2023-02-13 11:53:36,338 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 83e014aad99e479783ec460d80bc3761: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,339 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp/cf2/83e014aad99e479783ec460d80bc3761 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/83e014aad99e479783ec460d80bc3761 2023-02-13 11:53:36,342 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 00418fb274c5493684afe78e82550d77: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,343 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 83e014aad99e479783ec460d80bc3761: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,343 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29381611e213b47c8119c7022001ff40/cf1 of 29381611e213b47c8119c7022001ff40 into 00418fb274c5493684afe78e82550d77(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,343 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 29381611e213b47c8119c7022001ff40/cf2 of 29381611e213b47c8119c7022001ff40 into 83e014aad99e479783ec460d80bc3761(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,343 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29381611e213b47c8119c7022001ff40: 2023-02-13 11:53:36,343 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29381611e213b47c8119c7022001ff40: 2023-02-13 11:53:36,343 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40., storeName=29381611e213b47c8119c7022001ff40/cf1, priority=9, startTime=1676289216266; duration=0sec 2023-02-13 11:53:36,343 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40., storeName=29381611e213b47c8119c7022001ff40/cf2, priority=9, startTime=1676289216267; duration=0sec 2023-02-13 11:53:36,343 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,344 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,345 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-bottom 2023-02-13 11:53:36,350 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-bottom 2023-02-13 11:53:36,353 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-bottom 2023-02-13 11:53:36,356 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:36,356 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-bottom 2023-02-13 11:53:36,360 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017 2023-02-13 11:53:36,363 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863 2023-02-13 11:53:36,366 DEBUG [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632 2023-02-13 11:53:36,366 INFO [StoreOpener-34ef7c7eb6373dbebfbccd213918b582-1] regionserver.HStore(310): Store=34ef7c7eb6373dbebfbccd213918b582/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:36,367 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,368 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,370 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:53:36,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:36,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 34ef7c7eb6373dbebfbccd213918b582; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67517513, jitterRate=0.006089344620704651}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:53:36,371 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 34ef7c7eb6373dbebfbccd213918b582: Running coprocessor pre-open hook at 1676289216272Writing region info on filesystem at 1676289216272Initializing all the Stores at 1676289216273 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289216273Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289216274 (+1 ms)Cleaning up temporary data from old regions at 1676289216369 (+95 ms)Cleaning up detritus from prior splits at 1676289216369Running coprocessor post-open hooks at 1676289216371 (+2 ms)Region opened successfully at 1676289216371 2023-02-13 11:53:36,372 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582., pid=77, masterSystemTime=1676289216168 2023-02-13 11:53:36,372 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,373 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:36,373 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:36,373 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:53:36,373 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,373 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,373 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1912): 34ef7c7eb6373dbebfbccd213918b582/cf2 is initiating minor compaction (all files) 2023-02-13 11:53:36,373 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1912): 34ef7c7eb6373dbebfbccd213918b582/cf1 is initiating minor compaction (all files) 2023-02-13 11:53:36,373 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 34ef7c7eb6373dbebfbccd213918b582/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,373 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 34ef7c7eb6373dbebfbccd213918b582/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,374 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp, totalSize=35.0 K 2023-02-13 11:53:36,374 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp, totalSize=35.0 K 2023-02-13 11:53:36,374 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289212155 2023-02-13 11:53:36,374 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289212155 2023-02-13 11:53:36,374 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289212710 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289212710 2023-02-13 11:53:36,375 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289212969 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289212969 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289213260 2023-02-13 11:53:36,375 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=34ef7c7eb6373dbebfbccd213918b582, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289213260 2023-02-13 11:53:36,375 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289213563 2023-02-13 11:53:36,375 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289216375"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289216375"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289216375"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289216375"}]},"ts":"1676289216375"} 2023-02-13 11:53:36,376 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289213563 2023-02-13 11:53:36,376 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289213855 2023-02-13 11:53:36,376 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289213855 2023-02-13 11:53:36,376 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting 9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289214100 2023-02-13 11:53:36,376 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289214100 2023-02-13 11:53:36,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-02-13 11:53:36,379 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 34ef7c7eb6373dbebfbccd213918b582, server=jenkins-hbase16.apache.org,44743,1676289185212 in 361 msec 2023-02-13 11:53:36,380 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-02-13 11:53:36,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, ASSIGN in 518 msec 2023-02-13 11:53:36,383 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 in 1.4160 sec 2023-02-13 11:53:36,400 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] throttle.PressureAwareThroughputController(145): 34ef7c7eb6373dbebfbccd213918b582#cf1#compaction#74 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,401 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 34ef7c7eb6373dbebfbccd213918b582#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,419 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp/cf1/a8998125a0b34f779c574c47f2f66820 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/a8998125a0b34f779c574c47f2f66820 2023-02-13 11:53:36,419 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp/cf2/d494565785ef4818a0a80a0f16c12793 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/d494565785ef4818a0a80a0f16c12793 2023-02-13 11:53:36,424 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 34ef7c7eb6373dbebfbccd213918b582/cf2 of 34ef7c7eb6373dbebfbccd213918b582 into d494565785ef4818a0a80a0f16c12793(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,424 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 34ef7c7eb6373dbebfbccd213918b582/cf1 of 34ef7c7eb6373dbebfbccd213918b582 into a8998125a0b34f779c574c47f2f66820(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,425 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 34ef7c7eb6373dbebfbccd213918b582: 2023-02-13 11:53:36,425 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 34ef7c7eb6373dbebfbccd213918b582: 2023-02-13 11:53:36,425 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582., storeName=34ef7c7eb6373dbebfbccd213918b582/cf2, priority=9, startTime=1676289216373; duration=0sec 2023-02-13 11:53:36,425 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582., storeName=34ef7c7eb6373dbebfbccd213918b582/cf1, priority=9, startTime=1676289216372; duration=0sec 2023-02-13 11:53:36,425 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,425 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,493 DEBUG [Listener at localhost.localdomain/45781] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-02-13 11:53:36,497 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 188.40.62.62:47184, version=2.4.17-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-02-13 11:53:36,497 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,498 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 11:53:36,498 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HStore(1912): 34ef7c7eb6373dbebfbccd213918b582/cf1 is initiating major compaction (all files) 2023-02-13 11:53:36,498 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7118d04f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,498 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 34ef7c7eb6373dbebfbccd213918b582/cf1 in testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,498 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 11:53:36,498 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/a8998125a0b34f779c574c47f2f66820] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp, totalSize=6.8 K 2023-02-13 11:53:36,499 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HStore(1912): 34ef7c7eb6373dbebfbccd213918b582/cf2 is initiating major compaction (all files) 2023-02-13 11:53:36,499 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@2f285fac; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:36,499 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 34ef7c7eb6373dbebfbccd213918b582/cf2 in testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:36,499 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/d494565785ef4818a0a80a0f16c12793] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp, totalSize=6.8 K 2023-02-13 11:53:36,499 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] compactions.Compactor(207): Compacting a8998125a0b34f779c574c47f2f66820, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289212155 2023-02-13 11:53:36,499 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting d494565785ef4818a0a80a0f16c12793, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289212155 2023-02-13 11:53:36,501 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 11:53:36,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HStore(1912): 29381611e213b47c8119c7022001ff40/cf1 is initiating major compaction (all files) 2023-02-13 11:53:36,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5cabfbd7; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:36,501 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-02-13 11:53:36,502 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HStore(1912): 29381611e213b47c8119c7022001ff40/cf2 is initiating major compaction (all files) 2023-02-13 11:53:36,502 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@29f02cc8; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 11:53:36,502 INFO [Listener at localhost.localdomain/45781] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-02-13 11:53:36,508 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 34ef7c7eb6373dbebfbccd213918b582#cf2#compaction#76 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,516 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] throttle.PressureAwareThroughputController(145): 34ef7c7eb6373dbebfbccd213918b582#cf1#compaction#77 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,538 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:36,538 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:36,581 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp/cf2/fbd1bbb095be4f838d5328b9a9b12f7c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/fbd1bbb095be4f838d5328b9a9b12f7c 2023-02-13 11:53:36,591 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 34ef7c7eb6373dbebfbccd213918b582/cf2 of 34ef7c7eb6373dbebfbccd213918b582 into fbd1bbb095be4f838d5328b9a9b12f7c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,591 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 34ef7c7eb6373dbebfbccd213918b582: 2023-02-13 11:53:36,592 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676289216499; duration=0sec 2023-02-13 11:53:36,593 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-02-13 11:53:36,593 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 29381611e213b47c8119c7022001ff40/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,593 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/00418fb274c5493684afe78e82550d77] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp, totalSize=5.0 K 2023-02-13 11:53:36,594 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting 00418fb274c5493684afe78e82550d77, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,599 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 29381611e213b47c8119c7022001ff40#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,640 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd1a3a7cdbd84bb1b3c1ad2015e0c20b: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,642 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp/cf1/fd1a3a7cdbd84bb1b3c1ad2015e0c20b as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/fd1a3a7cdbd84bb1b3c1ad2015e0c20b 2023-02-13 11:53:36,652 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for fd1a3a7cdbd84bb1b3c1ad2015e0c20b: NONE, but ROW specified in column family configuration 2023-02-13 11:53:36,653 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:36,654 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:36,655 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 29381611e213b47c8119c7022001ff40/cf1 of 29381611e213b47c8119c7022001ff40 into fd1a3a7cdbd84bb1b3c1ad2015e0c20b(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,655 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29381611e213b47c8119c7022001ff40: 2023-02-13 11:53:36,655 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676289216501; duration=0sec 2023-02-13 11:53:36,656 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:53:36,656 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 29381611e213b47c8119c7022001ff40/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:36,656 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/83e014aad99e479783ec460d80bc3761] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp, totalSize=5.0 K 2023-02-13 11:53:36,657 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] compactions.Compactor(207): Compacting 83e014aad99e479783ec460d80bc3761, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-02-13 11:53:36,672 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] throttle.PressureAwareThroughputController(145): 29381611e213b47c8119c7022001ff40#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:53:36,762 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:36,763 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:36,871 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:36,871 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:36,988 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/.tmp/cf1/94b486eeb66348658239cc51a860764a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/94b486eeb66348658239cc51a860764a 2023-02-13 11:53:36,988 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:36,989 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:36,993 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 34ef7c7eb6373dbebfbccd213918b582/cf1 of 34ef7c7eb6373dbebfbccd213918b582 into 94b486eeb66348658239cc51a860764a(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:36,994 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 34ef7c7eb6373dbebfbccd213918b582: 2023-02-13 11:53:36,994 INFO [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1676289216498; duration=0sec 2023-02-13 11:53:36,994 DEBUG [RS:2;jenkins-hbase16:44743-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:37,105 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:37,105 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c21d0f16c9054c12837c9c93ab8db44a: NONE, but ROW specified in column family configuration 2023-02-13 11:53:37,106 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:37,106 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/.tmp/cf2/c21d0f16c9054c12837c9c93ab8db44a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/c21d0f16c9054c12837c9c93ab8db44a 2023-02-13 11:53:37,110 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for c21d0f16c9054c12837c9c93ab8db44a: NONE, but ROW specified in column family configuration 2023-02-13 11:53:37,111 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 29381611e213b47c8119c7022001ff40/cf2 of 29381611e213b47c8119c7022001ff40 into c21d0f16c9054c12837c9c93ab8db44a(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:53:37,111 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 29381611e213b47c8119c7022001ff40: 2023-02-13 11:53:37,111 INFO [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1676289216502; duration=0sec 2023-02-13 11:53:37,111 DEBUG [RS:2;jenkins-hbase16:44743-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:53:37,219 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. is not mergeable because it has references 2023-02-13 11:53:37,219 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=44743] regionserver.HRegion(1450): Region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. is not mergeable because it has references 2023-02-13 11:53:37,225 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-02-13 11:53:37,226 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:37,228 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,228 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,229 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 11:53:37,232 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289217232"}]},"ts":"1676289217232"} 2023-02-13 11:53:37,233 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-02-13 11:53:37,285 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-02-13 11:53:37,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, UNASSIGN}] 2023-02-13 11:53:37,288 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, UNASSIGN 2023-02-13 11:53:37,289 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=29381611e213b47c8119c7022001ff40, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:37,290 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289217289"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289217289"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289217289"}]},"ts":"1676289217289"} 2023-02-13 11:53:37,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 29381611e213b47c8119c7022001ff40, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:37,297 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, UNASSIGN 2023-02-13 11:53:37,298 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=34ef7c7eb6373dbebfbccd213918b582, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:37,298 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289217298"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289217298"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289217298"}]},"ts":"1676289217298"} 2023-02-13 11:53:37,299 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 34ef7c7eb6373dbebfbccd213918b582, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:37,333 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 11:53:37,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:37,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 29381611e213b47c8119c7022001ff40, disabling compactions & flushes 2023-02-13 11:53:37,447 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:37,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:37,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. after waiting 0 ms 2023-02-13 11:53:37,447 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:37,449 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/00418fb274c5493684afe78e82550d77, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938] to archive 2023-02-13 11:53:37,451 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:37,455 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,459 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,461 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,462 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,464 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:37,465 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:37,467 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/00418fb274c5493684afe78e82550d77 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/00418fb274c5493684afe78e82550d77 2023-02-13 11:53:37,468 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-83f5256fe6254166952803065fef9938 2023-02-13 11:53:37,470 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/83e014aad99e479783ec460d80bc3761, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5] to archive 2023-02-13 11:53:37,471 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:37,473 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,474 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,476 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,478 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,480 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:37,482 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:37,483 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/83e014aad99e479783ec460d80bc3761 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/83e014aad99e479783ec460d80bc3761 2023-02-13 11:53:37,485 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:37,491 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 11:53:37,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:37,492 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40. 2023-02-13 11:53:37,492 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 29381611e213b47c8119c7022001ff40: Waiting for close lock at 1676289217447Running coprocessor pre-close hooks at 1676289217447Disabling compacts and flushes for region at 1676289217447Disabling writes for close at 1676289217447Writing region close event to WAL at 1676289217487 (+40 ms)Running coprocessor post-close hooks at 1676289217492 (+5 ms)Closed at 1676289217492 2023-02-13 11:53:37,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 29381611e213b47c8119c7022001ff40 2023-02-13 11:53:37,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:37,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 34ef7c7eb6373dbebfbccd213918b582, disabling compactions & flushes 2023-02-13 11:53:37,494 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:37,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:37,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. after waiting 0 ms 2023-02-13 11:53:37,494 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:37,494 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=29381611e213b47c8119c7022001ff40, regionState=CLOSED 2023-02-13 11:53:37,495 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289217494"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289217494"}]},"ts":"1676289217494"} 2023-02-13 11:53:37,496 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/a8998125a0b34f779c574c47f2f66820, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c-bottom] to archive 2023-02-13 11:53:37,497 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:37,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-02-13 11:53:37,498 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 29381611e213b47c8119c7022001ff40, server=jenkins-hbase16.apache.org,44743,1676289185212 in 205 msec 2023-02-13 11:53:37,499 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/aa6a7f55aa5a4567815c088470a1e34c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,500 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=29381611e213b47c8119c7022001ff40, UNASSIGN in 212 msec 2023-02-13 11:53:37,500 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/72b2f4bd891e400ca142034de90354de.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,502 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/88a60af79f034c6ba19297ff86ae6c44.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,503 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-6db1be2e94604541bc57013dd4355d72 2023-02-13 11:53:37,504 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-5a745d2ca0194394bd9065d889694d16 2023-02-13 11:53:37,505 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-b550ed5cd8774bfd9415da720486c212 2023-02-13 11:53:37,506 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/a8998125a0b34f779c574c47f2f66820 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/a8998125a0b34f779c574c47f2f66820 2023-02-13 11:53:37,508 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,510 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/d494565785ef4818a0a80a0f16c12793, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa-bottom] to archive 2023-02-13 11:53:37,511 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:53:37,512 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/aa3c021f3ab4409a920f31854688350c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,513 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/a818e862481a4880be8cf5a731ea9d2c.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,515 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/cb89f41aa7c94824a9b0c33301ed6394.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,516 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-e8f667ea97404b249e323a8f4f73f632 2023-02-13 11:53:37,517 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-361d9e13c5e5430c80b1e8b7ca4ef017 2023-02-13 11:53:37,519 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/testSplitTableRegionDeletedRowsDaughter=1f69a2ae9ee6751a9ecdbbcfc72870d6-3f665a7c5f2f43a2b5fda71f86783863 2023-02-13 11:53:37,520 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/d494565785ef4818a0a80a0f16c12793 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/d494565785ef4818a0a80a0f16c12793 2023-02-13 11:53:37,521 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/dd152a59cc4b456d99d3287f9c93b3fa.1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-02-13 11:53:37,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:37,525 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582. 2023-02-13 11:53:37,525 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 34ef7c7eb6373dbebfbccd213918b582: Waiting for close lock at 1676289217494Running coprocessor pre-close hooks at 1676289217494Disabling compacts and flushes for region at 1676289217494Disabling writes for close at 1676289217494Writing region close event to WAL at 1676289217522 (+28 ms)Running coprocessor post-close hooks at 1676289217525 (+3 ms)Closed at 1676289217525 2023-02-13 11:53:37,527 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:37,528 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=34ef7c7eb6373dbebfbccd213918b582, regionState=CLOSED 2023-02-13 11:53:37,528 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1676289217528"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289217528"}]},"ts":"1676289217528"} 2023-02-13 11:53:37,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-02-13 11:53:37,531 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 34ef7c7eb6373dbebfbccd213918b582, server=jenkins-hbase16.apache.org,44743,1676289185212 in 230 msec 2023-02-13 11:53:37,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=80, resume processing ppid=79 2023-02-13 11:53:37,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=34ef7c7eb6373dbebfbccd213918b582, UNASSIGN in 244 msec 2023-02-13 11:53:37,533 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289217533"}]},"ts":"1676289217533"} 2023-02-13 11:53:37,534 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 11:53:37,534 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-02-13 11:53:37,543 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-02-13 11:53:37,545 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 316 msec 2023-02-13 11:53:37,836 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-02-13 11:53:37,837 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-02-13 11:53:37,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,841 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,844 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,845 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 11:53:37,853 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,853 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40 2023-02-13 11:53:37,853 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:37,855 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/recovered.edits] 2023-02-13 11:53:37,855 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/recovered.edits] 2023-02-13 11:53:37,855 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/recovered.edits] 2023-02-13 11:53:37,858 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/fd1a3a7cdbd84bb1b3c1ad2015e0c20b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf1/fd1a3a7cdbd84bb1b3c1ad2015e0c20b 2023-02-13 11:53:37,858 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/5a745d2ca0194394bd9065d889694d16 2023-02-13 11:53:37,858 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/94b486eeb66348658239cc51a860764a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf1/94b486eeb66348658239cc51a860764a 2023-02-13 11:53:37,860 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/6db1be2e94604541bc57013dd4355d72 2023-02-13 11:53:37,861 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/c21d0f16c9054c12837c9c93ab8db44a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/cf2/c21d0f16c9054c12837c9c93ab8db44a 2023-02-13 11:53:37,861 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/fbd1bbb095be4f838d5328b9a9b12f7c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/cf2/fbd1bbb095be4f838d5328b9a9b12f7c 2023-02-13 11:53:37,861 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/72b2f4bd891e400ca142034de90354de 2023-02-13 11:53:37,862 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/83f5256fe6254166952803065fef9938 2023-02-13 11:53:37,864 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/88a60af79f034c6ba19297ff86ae6c44 2023-02-13 11:53:37,864 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/recovered.edits/113.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582/recovered.edits/113.seqid 2023-02-13 11:53:37,864 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/recovered.edits/113.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40/recovered.edits/113.seqid 2023-02-13 11:53:37,865 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/34ef7c7eb6373dbebfbccd213918b582 2023-02-13 11:53:37,865 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/29381611e213b47c8119c7022001ff40 2023-02-13 11:53:37,866 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/9d07f74fcc0d49d0a3cb0ce76c11df5c 2023-02-13 11:53:37,867 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/aa6a7f55aa5a4567815c088470a1e34c 2023-02-13 11:53:37,869 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/b550ed5cd8774bfd9415da720486c212 2023-02-13 11:53:37,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/eea0bc4826424a578112120bb237b10d 2023-02-13 11:53:37,871 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf1/f1494265d50d4f1486bd7a80899f9453 2023-02-13 11:53:37,874 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/0e468b5ee6854258afa1a318fa16684d 2023-02-13 11:53:37,875 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/1910d209e53546a58c54e4e157f192d5 2023-02-13 11:53:37,877 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/24036b97aa0442c29cd6bed07023dae9 2023-02-13 11:53:37,878 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/361d9e13c5e5430c80b1e8b7ca4ef017 2023-02-13 11:53:37,879 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/3f665a7c5f2f43a2b5fda71f86783863 2023-02-13 11:53:37,880 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/a818e862481a4880be8cf5a731ea9d2c 2023-02-13 11:53:37,881 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/aa3c021f3ab4409a920f31854688350c 2023-02-13 11:53:37,881 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/cb89f41aa7c94824a9b0c33301ed6394 2023-02-13 11:53:37,882 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/dd152a59cc4b456d99d3287f9c93b3fa 2023-02-13 11:53:37,883 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/cf2/e8f667ea97404b249e323a8f4f73f632 2023-02-13 11:53:37,885 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/recovered.edits/106.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6/recovered.edits/106.seqid 2023-02-13 11:53:37,886 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/1f69a2ae9ee6751a9ecdbbcfc72870d6 2023-02-13 11:53:37,886 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-02-13 11:53:37,887 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,889 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-02-13 11:53:37,891 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-02-13 11:53:37,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,892 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-02-13 11:53:37,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289217892"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:37,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289217892"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:37,892 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289217892"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:37,894 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:53:37,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 1f69a2ae9ee6751a9ecdbbcfc72870d6, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676289211650.1f69a2ae9ee6751a9ecdbbcfc72870d6.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 34ef7c7eb6373dbebfbccd213918b582, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1676289214965.34ef7c7eb6373dbebfbccd213918b582.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 29381611e213b47c8119c7022001ff40, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1676289214965.29381611e213b47c8119c7022001ff40.', STARTKEY => '60', ENDKEY => ''}] 2023-02-13 11:53:37,894 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-02-13 11:53:37,894 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289217894"}]},"ts":"9223372036854775807"} 2023-02-13 11:53:37,895 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-02-13 11:53:37,927 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:53:37,928 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 88 msec 2023-02-13 11:53:37,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-02-13 11:53:37,947 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-02-13 11:53:37,967 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=414 (was 410) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37980 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:41072 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:41062 [Waiting for operation #7] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37990 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37694 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:41044 [Waiting for operation #8] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38032 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:41106 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37766 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:36222 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x61151f3a-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:36162 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:38046 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:36210 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-19 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:37976 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:41060 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1475625213_15 at /127.0.0.1:36202 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x2c6a7f26-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=850 (was 855), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=356 (was 308) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5079 (was 5227) 2023-02-13 11:53:37,982 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=414, OpenFileDescriptor=850, MaxFileDescriptor=60000, SystemLoadAverage=356, ProcessCount=169, AvailableMemoryMB=5078 2023-02-13 11:53:37,982 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:53:37,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:53:37,993 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:53:37,995 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:53:37,997 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:53:37,998 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:53:38,000 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,000 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 empty. 2023-02-13 11:53:38,001 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,001 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 11:53:38,011 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:38,011 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7671): creating {ENCODED => 0cd26988eceb006a09093a6068229c65, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1603): Closing 0cd26988eceb006a09093a6068229c65, disabling compactions & flushes 2023-02-13 11:53:38,021 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. after waiting 0 ms 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,021 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,021 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1557): Region close journal for 0cd26988eceb006a09093a6068229c65: Waiting for close lock at 1676289218021Disabling compacts and flushes for region at 1676289218021Disabling writes for close at 1676289218021Writing region close event to WAL at 1676289218021Closed at 1676289218021 2023-02-13 11:53:38,023 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:53:38,024 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218023"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289218023"}]},"ts":"1676289218023"} 2023-02-13 11:53:38,025 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:53:38,025 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:53:38,025 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289218025"}]},"ts":"1676289218025"} 2023-02-13 11:53:38,026 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-02-13 11:53:38,068 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:38,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:38,069 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:38,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:38,070 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:53:38,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, ASSIGN}] 2023-02-13 11:53:38,072 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, ASSIGN 2023-02-13 11:53:38,073 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:53:38,224 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:38,228 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:38,228 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218228"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289218228"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289218228"}]},"ts":"1676289218228"} 2023-02-13 11:53:38,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:38,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,391 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0cd26988eceb006a09093a6068229c65, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:38,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:38,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:38,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,392 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,395 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,399 DEBUG [StoreOpener-0cd26988eceb006a09093a6068229c65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:53:38,399 DEBUG [StoreOpener-0cd26988eceb006a09093a6068229c65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:53:38,399 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0cd26988eceb006a09093a6068229c65 columnFamilyName cf1 2023-02-13 11:53:38,400 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] regionserver.HStore(310): Store=0cd26988eceb006a09093a6068229c65/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:38,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,401 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,404 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,406 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:53:38,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 0cd26988eceb006a09093a6068229c65; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66140295, jitterRate=-0.01443280279636383}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 11:53:38,407 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 0cd26988eceb006a09093a6068229c65: Running coprocessor pre-open hook at 1676289218393Writing region info on filesystem at 1676289218393Initializing all the Stores at 1676289218395 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289218395Cleaning up temporary data from old regions at 1676289218402 (+7 ms)Cleaning up detritus from prior splits at 1676289218402Running coprocessor post-open hooks at 1676289218407 (+5 ms)Region opened successfully at 1676289218407 2023-02-13 11:53:38,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65., pid=87, masterSystemTime=1676289218383 2023-02-13 11:53:38,409 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,409 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:38,410 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218409"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289218409"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289218409"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289218409"}]},"ts":"1676289218409"} 2023-02-13 11:53:38,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-02-13 11:53:38,413 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121 in 180 msec 2023-02-13 11:53:38,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-02-13 11:53:38,415 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, ASSIGN in 343 msec 2023-02-13 11:53:38,415 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:53:38,416 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289218416"}]},"ts":"1676289218416"} 2023-02-13 11:53:38,417 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-02-13 11:53:38,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:53:38,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 433 msec 2023-02-13 11:53:38,502 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$12(2441): Client=jenkins//188.40.62.62 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-02-13 11:53:38,508 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:53:38,519 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 11:53:38,520 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-02-13 11:53:38,520 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-02-13 11:53:38,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-02-13 11:53:38,537 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, REOPEN/MOVE}] 2023-02-13 11:53:38,538 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, REOPEN/MOVE 2023-02-13 11:53:38,539 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:38,539 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218539"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289218539"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289218539"}]},"ts":"1676289218539"} 2023-02-13 11:53:38,540 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:38,693 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 0cd26988eceb006a09093a6068229c65, disabling compactions & flushes 2023-02-13 11:53:38,696 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. after waiting 0 ms 2023-02-13 11:53:38,696 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,703 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-02-13 11:53:38,704 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:53:38,705 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:38,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 0cd26988eceb006a09093a6068229c65: Waiting for close lock at 1676289218693Running coprocessor pre-close hooks at 1676289218693Disabling compacts and flushes for region at 1676289218693Disabling writes for close at 1676289218696 (+3 ms)Writing region close event to WAL at 1676289218697 (+1 ms)Running coprocessor post-close hooks at 1676289218704 (+7 ms)Closed at 1676289218705 (+1 ms) 2023-02-13 11:53:38,705 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(3509): Not adding moved region record: 0cd26988eceb006a09093a6068229c65 to self. 2023-02-13 11:53:38,707 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:38,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=CLOSED 2023-02-13 11:53:38,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218708"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289218708"}]},"ts":"1676289218708"} 2023-02-13 11:53:38,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-02-13 11:53:38,714 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121 in 171 msec 2023-02-13 11:53:38,715 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=true 2023-02-13 11:53:38,866 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:53:38,866 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:38,867 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289218866"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289218866"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289218866"}]},"ts":"1676289218866"} 2023-02-13 11:53:38,870 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:53:39,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:39,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 0cd26988eceb006a09093a6068229c65, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:53:39,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:39,032 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:39,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,033 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,035 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,037 DEBUG [StoreOpener-0cd26988eceb006a09093a6068229c65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:53:39,037 DEBUG [StoreOpener-0cd26988eceb006a09093a6068229c65-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:53:39,037 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0cd26988eceb006a09093a6068229c65 columnFamilyName cf1 2023-02-13 11:53:39,038 INFO [StoreOpener-0cd26988eceb006a09093a6068229c65-1] regionserver.HStore(310): Store=0cd26988eceb006a09093a6068229c65/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:53:39,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,041 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,044 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 0cd26988eceb006a09093a6068229c65 2023-02-13 11:53:39,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 0cd26988eceb006a09093a6068229c65; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62812002, jitterRate=-0.06402823328971863}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 11:53:39,046 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 0cd26988eceb006a09093a6068229c65: Running coprocessor pre-open hook at 1676289219033Writing region info on filesystem at 1676289219033Initializing all the Stores at 1676289219035 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289219035Cleaning up temporary data from old regions at 1676289219041 (+6 ms)Cleaning up detritus from prior splits at 1676289219042 (+1 ms)Running coprocessor post-open hooks at 1676289219046 (+4 ms)Region opened successfully at 1676289219046 2023-02-13 11:53:39,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65., pid=92, masterSystemTime=1676289219024 2023-02-13 11:53:39,048 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:39,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:53:39,049 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:53:39,049 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289219049"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289219049"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289219049"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289219049"}]},"ts":"1676289219049"} 2023-02-13 11:53:39,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-02-13 11:53:39,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121 in 180 msec 2023-02-13 11:53:39,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-02-13 11:53:39,054 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, REOPEN/MOVE in 515 msec 2023-02-13 11:53:39,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-02-13 11:53:39,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 532 msec 2023-02-13 11:53:39,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, ASSIGN}] 2023-02-13 11:53:39,060 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, ASSIGN 2023-02-13 11:53:39,061 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-02-13 11:53:39,211 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:53:39,212 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:53:39,212 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:53:39,212 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:53:39,212 DEBUG [jenkins-hbase16:44445] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-02-13 11:53:39,214 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=889c72465bac480c11b9d4e2bbc6fb8b, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:53:39,214 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289219214"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676289219214"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676289219214"}]},"ts":"1676289219214"} 2023-02-13 11:53:39,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 889c72465bac480c11b9d4e2bbc6fb8b, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:53:39,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:53:39,380 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 889c72465bac480c11b9d4e2bbc6fb8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-02-13 11:53:39,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:53:39,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:53:39,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:53:39,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:53:39,381 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:53:39,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:39,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:40,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,676 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:53:41,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:41,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,598 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-02-13 11:53:42,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:42,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,503 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:43,904 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,405 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:44,906 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,207 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,407 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:45,908 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,309 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,409 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,509 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,810 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:46,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,010 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,311 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,511 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:47,912 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,012 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,313 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,413 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,513 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,814 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:48,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,315 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,415 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,515 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,816 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:49,916 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,016 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,417 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,517 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:50,918 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,018 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,419 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:51,920 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,020 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,521 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,621 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:52,922 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,022 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:53,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:54,926 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:55,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:56,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:57,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:58,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:53:59,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:00,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:01,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:02,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:03,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,045 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,145 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,646 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,746 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:04,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,047 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,147 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,247 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,748 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:05,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,049 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,149 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,249 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,636 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 11:54:06,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,724 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 11:54:06,726 DEBUG [regionserver/jenkins-hbase16:0.Chore.1] throttle.PressureAwareCompactionThroughputController(103): CompactionPressure is 0.0, tune throughput to 50.00 MB/second 2023-02-13 11:54:06,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:06,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,151 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:07,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,554 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:08,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,055 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:09,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:10,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,058 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:11,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,060 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,561 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:12,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,463 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,864 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:13,964 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:14,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:15,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:16,971 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:17,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,875 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:18,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,276 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,376 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,777 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,877 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:19,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,278 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,378 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,779 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,879 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:20,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,280 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,781 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:21,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:22,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:23,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:24,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:25,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,189 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,289 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,690 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:26,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,291 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:27,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,193 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,694 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:28,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:29,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,197 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:30,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:31,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:32,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:33,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:34,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:35,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:36,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:37,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,214 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 11:54:38,715 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,717 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 11:54:38,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:38,918 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 11:54:39,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 11:54:39,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,721 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-02-13 11:54:39,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:39,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,723 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:40,724 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:40,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:40,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:40,725 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:40,727 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 0cd26988eceb006a09093a6068229c65, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:40,729 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f exec-time=6 msec 2023-02-13 11:54:40,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:40,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-02-13 11:54:41,020 INFO [StoreOpener-889c72465bac480c11b9d4e2bbc6fb8b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:54:41,024 DEBUG [StoreOpener-889c72465bac480c11b9d4e2bbc6fb8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:54:41,024 DEBUG [StoreOpener-889c72465bac480c11b9d4e2bbc6fb8b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1 2023-02-13 11:54:41,024 INFO [StoreOpener-889c72465bac480c11b9d4e2bbc6fb8b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 889c72465bac480c11b9d4e2bbc6fb8b columnFamilyName cf1 2023-02-13 11:54:41,025 INFO [StoreOpener-889c72465bac480c11b9d4e2bbc6fb8b-1] regionserver.HStore(310): Store=889c72465bac480c11b9d4e2bbc6fb8b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:41,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 889c72465bac480c11b9d4e2bbc6fb8b; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69756306, jitterRate=0.03944995999336243}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-02-13 11:54:41,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 889c72465bac480c11b9d4e2bbc6fb8b: Running coprocessor pre-open hook at 1676289219381Initializing all the Stores at 1676289281019 (+61638 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289281019Running coprocessor post-open hooks at 1676289281027 (+8 ms)Region opened successfully at 1676289281027 2023-02-13 11:54:41,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b., pid=94, masterSystemTime=1676289219377 2023-02-13 11:54:41,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:41,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:41,028 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=889c72465bac480c11b9d4e2bbc6fb8b, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:41,029 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289281028"},{"qualifier":"server_0001","vlen":32,"tag":[],"timestamp":"1676289281028"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1676289281028"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1676289281028"}]},"ts":"1676289281028"} 2023-02-13 11:54:41,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-02-13 11:54:41,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 889c72465bac480c11b9d4e2bbc6fb8b, server=jenkins-hbase16.apache.org,44743,1676289185212 in 1 mins, 1.808 sec 2023-02-13 11:54:41,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-02-13 11:54:41,033 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, ASSIGN in 1 mins, 1.973 sec 2023-02-13 11:54:41,035 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.53 sec 2023-02-13 11:54:41,035 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1676289218500","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62535,"client":"188.40.62.62:36342","queuetimems":0,"class":"HMaster"} 2023-02-13 11:54:41,035 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 656, skipping over this response of 3 bytes 2023-02-13 11:54:42,975 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up 2023-02-13 11:54:42,976 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:42,978 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:42,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:42,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:42,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 11:54:42,982 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289282982"}]},"ts":"1676289282982"} 2023-02-13 11:54:42,983 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-02-13 11:54:43,024 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-02-13 11:54:43,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, UNASSIGN}] 2023-02-13 11:54:43,026 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, UNASSIGN 2023-02-13 11:54:43,026 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=889c72465bac480c11b9d4e2bbc6fb8b, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,027 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289283026"},{"qualifier":"sn_0001","vlen":46,"tag":[],"timestamp":"1676289283026"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1676289283026"}]},"ts":"1676289283026"} 2023-02-13 11:54:43,028 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 889c72465bac480c11b9d4e2bbc6fb8b, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:54:43,030 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, UNASSIGN 2023-02-13 11:54:43,031 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:43,031 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289283031"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289283031"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289283031"}]},"ts":"1676289283031"} 2023-02-13 11:54:43,032 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:43,083 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 11:54:43,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:54:43,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 889c72465bac480c11b9d4e2bbc6fb8b, disabling compactions & flushes 2023-02-13 11:54:43,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:43,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:43,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. after waiting 0 ms 2023-02-13 11:54:43,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:43,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:43,182 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b. 2023-02-13 11:54:43,182 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 889c72465bac480c11b9d4e2bbc6fb8b: Waiting for close lock at 1676289283180Running coprocessor pre-close hooks at 1676289283180Disabling compacts and flushes for region at 1676289283180Disabling writes for close at 1676289283181 (+1 ms)Writing region close event to WAL at 1676289283182 (+1 ms)Running coprocessor post-close hooks at 1676289283182Closed at 1676289283182 2023-02-13 11:54:43,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 0cd26988eceb006a09093a6068229c65 2023-02-13 11:54:43,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 0cd26988eceb006a09093a6068229c65, disabling compactions & flushes 2023-02-13 11:54:43,184 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:54:43,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:54:43,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. after waiting 0 ms 2023-02-13 11:54:43,184 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:54:43,185 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 889c72465bac480c11b9d4e2bbc6fb8b 2023-02-13 11:54:43,185 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=889c72465bac480c11b9d4e2bbc6fb8b, regionState=CLOSED 2023-02-13 11:54:43,186 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289283185"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1676289283185"}]},"ts":"1676289283185"} 2023-02-13 11:54:43,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-02-13 11:54:43,189 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:43,190 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65. 2023-02-13 11:54:43,190 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 0cd26988eceb006a09093a6068229c65: Waiting for close lock at 1676289283183Running coprocessor pre-close hooks at 1676289283183Disabling compacts and flushes for region at 1676289283183Disabling writes for close at 1676289283184 (+1 ms)Writing region close event to WAL at 1676289283185 (+1 ms)Running coprocessor post-close hooks at 1676289283189 (+4 ms)Closed at 1676289283190 (+1 ms) 2023-02-13 11:54:43,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-02-13 11:54:43,190 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 889c72465bac480c11b9d4e2bbc6fb8b, server=jenkins-hbase16.apache.org,44743,1676289185212 in 160 msec 2023-02-13 11:54:43,191 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 0cd26988eceb006a09093a6068229c65 2023-02-13 11:54:43,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b, UNASSIGN in 165 msec 2023-02-13 11:54:43,192 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=0cd26988eceb006a09093a6068229c65, regionState=CLOSED 2023-02-13 11:54:43,192 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1676289283192"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289283192"}]},"ts":"1676289283192"} 2023-02-13 11:54:43,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-02-13 11:54:43,194 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 0cd26988eceb006a09093a6068229c65, server=jenkins-hbase16.apache.org,43159,1676289185121 in 161 msec 2023-02-13 11:54:43,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-02-13 11:54:43,200 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=0cd26988eceb006a09093a6068229c65, UNASSIGN in 169 msec 2023-02-13 11:54:43,201 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289283201"}]},"ts":"1676289283201"} 2023-02-13 11:54:43,202 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-02-13 11:54:43,207 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-02-13 11:54:43,209 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 229 msec 2023-02-13 11:54:43,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-02-13 11:54:43,285 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-02-13 11:54:43,285 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,287 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,288 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,288 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 11:54:43,292 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:54:43,294 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits] 2023-02-13 11:54:43,298 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits/7.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65/recovered.edits/7.seqid 2023-02-13 11:54:43,299 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/0cd26988eceb006a09093a6068229c65 2023-02-13 11:54:43,299 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-02-13 11:54:43,301 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,302 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-02-13 11:54:43,304 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-02-13 11:54:43,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-02-13 11:54:43,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289283305"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:43,305 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289283305"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:43,307 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-02-13 11:54:43,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0cd26988eceb006a09093a6068229c65, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994.0cd26988eceb006a09093a6068229c65.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 889c72465bac480c11b9d4e2bbc6fb8b, NAME => 'testRollbackForSplitTableRegionWithReplica,,1676289217994_0001.889c72465bac480c11b9d4e2bbc6fb8b.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-02-13 11:54:43,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-02-13 11:54:43,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289283307"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:43,308 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-02-13 11:54:43,316 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:43,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 31 msec 2023-02-13 11:54:43,390 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-02-13 11:54:43,390 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-02-13 11:54:43,401 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 414), OpenFileDescriptor=658 (was 850), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=199 (was 356), ProcessCount=169 (was 169), AvailableMemoryMB=5352 (was 5078) - AvailableMemoryMB LEAK? - 2023-02-13 11:54:43,413 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=199, ProcessCount=169, AvailableMemoryMB=5351 2023-02-13 11:54:43,413 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:43,424 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:54:43,424 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:54:43,426 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:43,428 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:54:43,428 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:54:43,430 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,430 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 empty. 2023-02-13 11:54:43,431 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,431 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 11:54:43,445 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-02-13 11:54:43,446 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7671): creating {ENCODED => 29707658d84657d657a0aead66d854d1, NAME => 'testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1603): Closing 29707658d84657d657a0aead66d854d1, disabling compactions & flushes 2023-02-13 11:54:43,455 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. after waiting 0 ms 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,455 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,455 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1557): Region close journal for 29707658d84657d657a0aead66d854d1: Waiting for close lock at 1676289283455Disabling compacts and flushes for region at 1676289283455Disabling writes for close at 1676289283455Writing region close event to WAL at 1676289283455Closed at 1676289283455 2023-02-13 11:54:43,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:54:43,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676289283457"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289283457"}]},"ts":"1676289283457"} 2023-02-13 11:54:43,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:54:43,459 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:54:43,459 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289283459"}]},"ts":"1676289283459"} 2023-02-13 11:54:43,460 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-02-13 11:54:43,480 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:54:43,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:54:43,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:54:43,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:54:43,481 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:54:43,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, ASSIGN}] 2023-02-13 11:54:43,483 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, ASSIGN 2023-02-13 11:54:43,483 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44743,1676289185212; forceNewPlan=false, retain=false 2023-02-13 11:54:43,634 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:54:43,635 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=29707658d84657d657a0aead66d854d1, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,635 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676289283635"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289283635"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289283635"}]},"ts":"1676289283635"} 2023-02-13 11:54:43,637 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 29707658d84657d657a0aead66d854d1, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:54:43,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 29707658d84657d657a0aead66d854d1, NAME => 'testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:54:43,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:43,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:43,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,795 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,796 DEBUG [StoreOpener-29707658d84657d657a0aead66d854d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1 2023-02-13 11:54:43,796 DEBUG [StoreOpener-29707658d84657d657a0aead66d854d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1 2023-02-13 11:54:43,796 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29707658d84657d657a0aead66d854d1 columnFamilyName cf1 2023-02-13 11:54:43,797 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] regionserver.HStore(310): Store=29707658d84657d657a0aead66d854d1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:43,797 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,798 DEBUG [StoreOpener-29707658d84657d657a0aead66d854d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2 2023-02-13 11:54:43,798 DEBUG [StoreOpener-29707658d84657d657a0aead66d854d1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2 2023-02-13 11:54:43,799 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 29707658d84657d657a0aead66d854d1 columnFamilyName cf2 2023-02-13 11:54:43,799 INFO [StoreOpener-29707658d84657d657a0aead66d854d1-1] regionserver.HStore(310): Store=29707658d84657d657a0aead66d854d1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:43,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,800 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,802 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:43,803 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:43,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:54:43,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 29707658d84657d657a0aead66d854d1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74455527, jitterRate=0.1094738095998764}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:43,806 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 29707658d84657d657a0aead66d854d1: Running coprocessor pre-open hook at 1676289283793Writing region info on filesystem at 1676289283793Initializing all the Stores at 1676289283794 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289283794Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289283794Cleaning up temporary data from old regions at 1676289283801 (+7 ms)Cleaning up detritus from prior splits at 1676289283801Running coprocessor post-open hooks at 1676289283806 (+5 ms)Region opened successfully at 1676289283806 2023-02-13 11:54:43,806 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1., pid=104, masterSystemTime=1676289283788 2023-02-13 11:54:43,808 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,808 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,808 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=29707658d84657d657a0aead66d854d1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,808 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676289283808"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289283808"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289283808"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289283808"}]},"ts":"1676289283808"} 2023-02-13 11:54:43,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-02-13 11:54:43,811 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 29707658d84657d657a0aead66d854d1, server=jenkins-hbase16.apache.org,44743,1676289185212 in 173 msec 2023-02-13 11:54:43,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-02-13 11:54:43,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, ASSIGN in 330 msec 2023-02-13 11:54:43,813 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:54:43,813 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289283813"}]},"ts":"1676289283813"} 2023-02-13 11:54:43,814 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-02-13 11:54:43,841 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:54:43,842 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 416 msec 2023-02-13 11:54:43,934 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:43,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:43,936 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:43,936 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:43,936 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:43,936 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:43,936 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:43,936 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,946 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:43,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:43,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:43,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:43,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:43,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:43,947 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,947 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:43,947 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:43,948 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:43,949 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,948 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:43,949 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:43,949 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:43,950 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,957 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:43,963 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/6d7d6e4426584fe48f93413ff87f68aa 2023-02-13 11:54:43,975 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/c1da89dddac94347adec2abafd332a29 2023-02-13 11:54:43,979 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/6d7d6e4426584fe48f93413ff87f68aa as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/6d7d6e4426584fe48f93413ff87f68aa 2023-02-13 11:54:43,983 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/6d7d6e4426584fe48f93413ff87f68aa, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:43,984 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/c1da89dddac94347adec2abafd332a29 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/c1da89dddac94347adec2abafd332a29 2023-02-13 11:54:43,988 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/c1da89dddac94347adec2abafd332a29, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:43,989 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 39ms, sequenceid=6, compaction requested=false 2023-02-13 11:54:43,989 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-02-13 11:54:43,989 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:43,989 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:43,989 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:43,999 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,999 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:43,999 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:43,999 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:43,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:44,001 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2037c3b1[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:44,001 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:44,001 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,007 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,008 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:44,007 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,008 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,008 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:44,015 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,015 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,015 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,024 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:44,028 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:44,028 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:44,028 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:44,028 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:44,028 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:44,028 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,038 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,039 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,039 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,039 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,039 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,041 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,041 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,041 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,041 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:44,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:44,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,049 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:44,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:44,049 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,050 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,050 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,150 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:44,151 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:44,162 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:44,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:44,164 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:44,164 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:44,164 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,164 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:44,164 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:44,164 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,164 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,194 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,194 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,194 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,194 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,195 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,195 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,195 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,196 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,196 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,196 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,197 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:44,197 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:44,197 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,197 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:44,197 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:44,205 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/7a10977d1edb4a24a4aa15728a01c91e 2023-02-13 11:54:44,207 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,215 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,215 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,215 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,215 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,216 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,217 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/ba0989df4fed4bbea40b3f931b29ae9d 2023-02-13 11:54:44,221 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/7a10977d1edb4a24a4aa15728a01c91e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e 2023-02-13 11:54:44,225 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:44,226 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/ba0989df4fed4bbea40b3f931b29ae9d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d 2023-02-13 11:54:44,230 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:44,231 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 34ms, sequenceid=19, compaction requested=false 2023-02-13 11:54:44,231 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:44,231 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,231 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:44,231 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:44,231 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,231 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,231 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,240 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,240 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,242 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:44,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a525940[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:44,243 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:44,243 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,249 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,249 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:44,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,257 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,258 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:44,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:44,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:44,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:44,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:44,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:44,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,265 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,265 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,266 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,266 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,266 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,267 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,268 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,268 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,269 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:44,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:44,280 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,280 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,281 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,381 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:44,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:44,394 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:44,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:44,395 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:44,396 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:44,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:44,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:44,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,405 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,405 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,405 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,406 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:44,407 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,407 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,407 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:44,407 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:44,415 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,415 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,416 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,416 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,416 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,416 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,416 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/1a2a21f4c84d4ea9b8f823a65552e70a 2023-02-13 11:54:44,430 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/1f7f0905f92b4b029d92044ac773f6ce 2023-02-13 11:54:44,434 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/1a2a21f4c84d4ea9b8f823a65552e70a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/1a2a21f4c84d4ea9b8f823a65552e70a 2023-02-13 11:54:44,438 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/1a2a21f4c84d4ea9b8f823a65552e70a, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:44,439 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/1f7f0905f92b4b029d92044ac773f6ce as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/1f7f0905f92b4b029d92044ac773f6ce 2023-02-13 11:54:44,443 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/1f7f0905f92b4b029d92044ac773f6ce, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:44,444 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 37ms, sequenceid=32, compaction requested=true 2023-02-13 11:54:44,444 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:44,444 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,444 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:44,444 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:44,444 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,444 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,444 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,455 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,455 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:44,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67b656c0[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:44,457 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:44,457 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,465 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,465 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,465 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,465 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,466 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:44,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:44,466 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:44,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,482 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,482 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:44,485 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:44,485 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:44,485 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:44,485 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:44,485 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:44,485 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,492 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,493 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,493 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,493 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:44,525 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:44,525 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,525 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,525 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,526 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,626 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:44,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:44,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:44,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:44,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:44,641 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:44,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:44,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:44,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,655 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,655 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,655 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:44,655 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,656 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:44,657 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,657 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,657 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,665 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,666 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/3eaadc02e7054fceaff21fe357a8cecb 2023-02-13 11:54:44,674 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,674 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,674 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,674 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,674 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,676 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/01719ad3145942228ab3f506bf657116 2023-02-13 11:54:44,680 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/3eaadc02e7054fceaff21fe357a8cecb as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/3eaadc02e7054fceaff21fe357a8cecb 2023-02-13 11:54:44,684 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/3eaadc02e7054fceaff21fe357a8cecb, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:44,685 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/01719ad3145942228ab3f506bf657116 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/01719ad3145942228ab3f506bf657116 2023-02-13 11:54:44,688 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/01719ad3145942228ab3f506bf657116, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:44,689 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 32ms, sequenceid=45, compaction requested=true 2023-02-13 11:54:44,689 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:44,689 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:44,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:44,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,699 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,699 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:44,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7db81409[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:44,701 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:44,701 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,707 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:44,707 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,707 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,707 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,708 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,708 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:44,707 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:44,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:44,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:44,708 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:44,715 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,715 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,715 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,724 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:44,724 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:44,727 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:44,727 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:44,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:44,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:44,727 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:44,727 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,733 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,733 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,733 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,733 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,734 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,734 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,735 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,735 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,735 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:44,747 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,746 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,746 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,747 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:44,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,747 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,748 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:44,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:44,747 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,748 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,748 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,748 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,748 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,847 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:44,849 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:44,859 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:44,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:44,861 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:44,861 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:44,861 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,861 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:44,861 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:44,861 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,861 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,871 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,871 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:44,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,872 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:44,872 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:44,873 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,873 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:44,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,873 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:44,873 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:44,873 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:44,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,874 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:44,882 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,884 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/b65c14c495044b7e90ba4ab22417643c 2023-02-13 11:54:44,890 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,890 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,890 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,898 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/06b8e810180d46bda0e41524eaf285f9 2023-02-13 11:54:44,902 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/b65c14c495044b7e90ba4ab22417643c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/b65c14c495044b7e90ba4ab22417643c 2023-02-13 11:54:44,906 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/b65c14c495044b7e90ba4ab22417643c, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:44,906 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/06b8e810180d46bda0e41524eaf285f9 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/06b8e810180d46bda0e41524eaf285f9 2023-02-13 11:54:44,911 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/06b8e810180d46bda0e41524eaf285f9, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:44,912 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 38ms, sequenceid=58, compaction requested=true 2023-02-13 11:54:44,912 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:44,912 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:44,912 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:44,912 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:44,912 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,912 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:44,912 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:44,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,946 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:44,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:44,946 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:44,947 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:44,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:44,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:44,949 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@638c6a44[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:44,949 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:44,949 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,013 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:45,013 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:45,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,014 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,014 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,014 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,032 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,032 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,032 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:45,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:45,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:45,036 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:45,036 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:45,036 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:45,036 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:45,036 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:45,036 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,047 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,048 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,049 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,049 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:45,049 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:45,057 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:45,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,057 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,057 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,057 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:45,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,058 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,058 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,158 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:45,159 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:45,170 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:45,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:45,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-02-13 11:54:45,171 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-02-13 11:54:45,172 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:45,172 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-02-13 11:54:45,172 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-02-13 11:54:45,172 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,172 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,182 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,182 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,182 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,183 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:45,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,183 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:45,184 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-02-13 11:54:45,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:45,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:45,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:45,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:45,184 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:45,184 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:45,184 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:45,184 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,185 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. started... 2023-02-13 11:54:45,185 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:45,190 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,190 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,191 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,191 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:45,191 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,191 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:45,194 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/c8cb913b455b4bfa9bc546f0c8dd0180 2023-02-13 11:54:45,205 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/611146950d2048589270316b0442b7c4 2023-02-13 11:54:45,209 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/c8cb913b455b4bfa9bc546f0c8dd0180 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/c8cb913b455b4bfa9bc546f0c8dd0180 2023-02-13 11:54:45,213 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/c8cb913b455b4bfa9bc546f0c8dd0180, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:45,214 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/611146950d2048589270316b0442b7c4 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/611146950d2048589270316b0442b7c4 2023-02-13 11:54:45,218 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/611146950d2048589270316b0442b7c4, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:45,219 INFO [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 33ms, sequenceid=71, compaction requested=true 2023-02-13 11:54:45,219 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:45,219 DEBUG [rs(jenkins-hbase16.apache.org,44743,1676289185212)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,219 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:45,219 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:45,219 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,219 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-02-13 11:54:45,219 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-02-13 11:54:45,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,230 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:45,230 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:45,230 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:45,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-02-13 11:54:45,232 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6b9cb72c[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:45,232 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-02-13 11:54:45,232 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,240 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,240 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,240 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,241 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-02-13 11:54:45,241 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,249 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,249 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:45,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:45,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-02-13 11:54:45,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,257 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-02-13 11:54:45,257 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:45,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:45,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase16.apache.org,44743,1676289185212': 2023-02-13 11:54:45,260 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-02-13 11:54:45,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-02-13 11:54:45,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:45,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:45,260 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-02-13 11:54:45,260 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:45,271 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,271 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,271 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,271 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:45,271 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,271 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,272 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,272 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,273 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,273 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,274 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:45,274 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-02-13 11:54:45,274 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-02-13 11:54:45,282 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,282 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:45,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:45,282 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:45,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:45,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-02-13 11:54:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,283 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:45,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-02-13 11:54:45,284 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-02-13 11:54:45,383 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:45,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:45,392 INFO [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,393 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e because midkey is the same as first or last row 2023-02-13 11:54:45,393 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d because midkey is the same as first or last row 2023-02-13 11:54:45,393 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(2744): Flushing 29707658d84657d657a0aead66d854d1 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:54:45,403 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/df2bb87cea9042308b211ba6a3bc69bd 2023-02-13 11:54:45,413 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/dea52578eaeb4491b4014614fd6ddf8b 2023-02-13 11:54:45,417 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf1/df2bb87cea9042308b211ba6a3bc69bd as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/df2bb87cea9042308b211ba6a3bc69bd 2023-02-13 11:54:45,420 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/df2bb87cea9042308b211ba6a3bc69bd, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:45,421 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/.tmp/cf2/dea52578eaeb4491b4014614fd6ddf8b as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/dea52578eaeb4491b4014614fd6ddf8b 2023-02-13 11:54:45,425 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/dea52578eaeb4491b4014614fd6ddf8b, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:45,426 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 29707658d84657d657a0aead66d854d1 in 33ms, sequenceid=82, compaction requested=true 2023-02-13 11:54:45,426 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.HRegion(2445): Flush status journal for 29707658d84657d657a0aead66d854d1: 2023-02-13 11:54:45,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e because midkey is the same as first or last row 2023-02-13 11:54:45,427 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=44743] regionserver.StoreUtils(129): cannot split hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d because midkey is the same as first or last row 2023-02-13 11:54:45,427 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,427 DEBUG [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-02-13 11:54:45,427 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:45,429 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-02-13 11:54:45,429 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testInvalidSplitKey 2023-02-13 11:54:45,430 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289285432"}]},"ts":"1676289285432"} 2023-02-13 11:54:45,432 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 11:54:45,433 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-02-13 11:54:45,440 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-02-13 11:54:45,441 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, UNASSIGN}] 2023-02-13 11:54:45,442 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, UNASSIGN 2023-02-13 11:54:45,443 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=29707658d84657d657a0aead66d854d1, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:45,443 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676289285443"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289285443"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289285443"}]},"ts":"1676289285443"} 2023-02-13 11:54:45,444 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 29707658d84657d657a0aead66d854d1, server=jenkins-hbase16.apache.org,44743,1676289185212}] 2023-02-13 11:54:45,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 11:54:45,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 29707658d84657d657a0aead66d854d1, disabling compactions & flushes 2023-02-13 11:54:45,596 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. after waiting 0 ms 2023-02-13 11:54:45,596 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:54:45,600 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:45,601 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1. 2023-02-13 11:54:45,601 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 29707658d84657d657a0aead66d854d1: Waiting for close lock at 1676289285596Running coprocessor pre-close hooks at 1676289285596Disabling compacts and flushes for region at 1676289285596Disabling writes for close at 1676289285596Writing region close event to WAL at 1676289285597 (+1 ms)Running coprocessor post-close hooks at 1676289285600 (+3 ms)Closed at 1676289285601 (+1 ms) 2023-02-13 11:54:45,602 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 29707658d84657d657a0aead66d854d1 2023-02-13 11:54:45,602 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=29707658d84657d657a0aead66d854d1, regionState=CLOSED 2023-02-13 11:54:45,602 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1676289285602"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289285602"}]},"ts":"1676289285602"} 2023-02-13 11:54:45,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-02-13 11:54:45,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 29707658d84657d657a0aead66d854d1, server=jenkins-hbase16.apache.org,44743,1676289185212 in 159 msec 2023-02-13 11:54:45,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-02-13 11:54:45,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=29707658d84657d657a0aead66d854d1, UNASSIGN in 163 msec 2023-02-13 11:54:45,606 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289285606"}]},"ts":"1676289285606"} 2023-02-13 11:54:45,607 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-02-13 11:54:45,615 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-02-13 11:54:45,617 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 187 msec 2023-02-13 11:54:45,735 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-02-13 11:54:45,735 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-02-13 11:54:45,736 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testInvalidSplitKey 2023-02-13 11:54:45,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,738 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,739 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,739 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 11:54:45,742 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:45,743 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/recovered.edits] 2023-02-13 11:54:45,747 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/1a2a21f4c84d4ea9b8f823a65552e70a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/1a2a21f4c84d4ea9b8f823a65552e70a 2023-02-13 11:54:45,748 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/3eaadc02e7054fceaff21fe357a8cecb to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/3eaadc02e7054fceaff21fe357a8cecb 2023-02-13 11:54:45,749 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/6d7d6e4426584fe48f93413ff87f68aa to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/6d7d6e4426584fe48f93413ff87f68aa 2023-02-13 11:54:45,750 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/7a10977d1edb4a24a4aa15728a01c91e 2023-02-13 11:54:45,751 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/b65c14c495044b7e90ba4ab22417643c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/b65c14c495044b7e90ba4ab22417643c 2023-02-13 11:54:45,752 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/c8cb913b455b4bfa9bc546f0c8dd0180 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/c8cb913b455b4bfa9bc546f0c8dd0180 2023-02-13 11:54:45,753 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/df2bb87cea9042308b211ba6a3bc69bd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf1/df2bb87cea9042308b211ba6a3bc69bd 2023-02-13 11:54:45,755 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/01719ad3145942228ab3f506bf657116 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/01719ad3145942228ab3f506bf657116 2023-02-13 11:54:45,755 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/06b8e810180d46bda0e41524eaf285f9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/06b8e810180d46bda0e41524eaf285f9 2023-02-13 11:54:45,756 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/1f7f0905f92b4b029d92044ac773f6ce to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/1f7f0905f92b4b029d92044ac773f6ce 2023-02-13 11:54:45,757 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/611146950d2048589270316b0442b7c4 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/611146950d2048589270316b0442b7c4 2023-02-13 11:54:45,758 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/ba0989df4fed4bbea40b3f931b29ae9d 2023-02-13 11:54:45,759 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/c1da89dddac94347adec2abafd332a29 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/c1da89dddac94347adec2abafd332a29 2023-02-13 11:54:45,760 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/dea52578eaeb4491b4014614fd6ddf8b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/cf2/dea52578eaeb4491b4014614fd6ddf8b 2023-02-13 11:54:45,761 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1/recovered.edits/85.seqid 2023-02-13 11:54:45,762 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testInvalidSplitKey/29707658d84657d657a0aead66d854d1 2023-02-13 11:54:45,762 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-02-13 11:54:45,764 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,765 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-02-13 11:54:45,768 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-02-13 11:54:45,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-02-13 11:54:45,770 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289285770"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:45,771 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-02-13 11:54:45,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 29707658d84657d657a0aead66d854d1, NAME => 'testInvalidSplitKey,,1676289283425.29707658d84657d657a0aead66d854d1.', STARTKEY => '', ENDKEY => ''}] 2023-02-13 11:54:45,771 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-02-13 11:54:45,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289285771"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:45,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-02-13 11:54:45,783 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:45,784 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 47 msec 2023-02-13 11:54:45,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-02-13 11:54:45,841 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-02-13 11:54:45,853 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=370 (was 348) - Thread LEAK? -, OpenFileDescriptor=747 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=199 (was 199), ProcessCount=169 (was 169), AvailableMemoryMB=5348 (was 5351) 2023-02-13 11:54:45,864 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=370, OpenFileDescriptor=747, MaxFileDescriptor=60000, SystemLoadAverage=199, ProcessCount=169, AvailableMemoryMB=5347 2023-02-13 11:54:45,865 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:45,874 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:54:45,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:54:45,876 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:45,877 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:54:45,878 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:54:45,879 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:45,880 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b empty. 2023-02-13 11:54:45,880 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:45,880 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 11:54:45,895 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-02-13 11:54:45,896 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7671): creating {ENCODED => acfc991c22a20180f5b87d31411aa95b, NAME => 'testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1603): Closing acfc991c22a20180f5b87d31411aa95b, disabling compactions & flushes 2023-02-13 11:54:45,905 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. after waiting 0 ms 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:45,905 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:45,905 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1557): Region close journal for acfc991c22a20180f5b87d31411aa95b: Waiting for close lock at 1676289285905Disabling compacts and flushes for region at 1676289285905Disabling writes for close at 1676289285905Writing region close event to WAL at 1676289285905Closed at 1676289285905 2023-02-13 11:54:45,907 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:54:45,907 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289285907"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289285907"}]},"ts":"1676289285907"} 2023-02-13 11:54:45,908 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:54:45,909 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:54:45,909 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289285909"}]},"ts":"1676289285909"} 2023-02-13 11:54:45,910 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-02-13 11:54:45,924 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:54:45,925 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:54:45,925 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:54:45,925 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:54:45,925 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:54:45,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, ASSIGN}] 2023-02-13 11:54:45,927 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, ASSIGN 2023-02-13 11:54:45,927 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:54:46,077 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:54:46,079 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=acfc991c22a20180f5b87d31411aa95b, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,079 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289286078"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289286078"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289286078"}]},"ts":"1676289286078"} 2023-02-13 11:54:46,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:46,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => acfc991c22a20180f5b87d31411aa95b, NAME => 'testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:54:46,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:46,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:46,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,237 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,238 DEBUG [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1 2023-02-13 11:54:46,238 DEBUG [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1 2023-02-13 11:54:46,238 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region acfc991c22a20180f5b87d31411aa95b columnFamilyName cf1 2023-02-13 11:54:46,239 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] regionserver.HStore(310): Store=acfc991c22a20180f5b87d31411aa95b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:46,239 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,240 DEBUG [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2 2023-02-13 11:54:46,240 DEBUG [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2 2023-02-13 11:54:46,240 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region acfc991c22a20180f5b87d31411aa95b columnFamilyName cf2 2023-02-13 11:54:46,241 INFO [StoreOpener-acfc991c22a20180f5b87d31411aa95b-1] regionserver.HStore(310): Store=acfc991c22a20180f5b87d31411aa95b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:46,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,242 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,243 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:46,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:46,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:54:46,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened acfc991c22a20180f5b87d31411aa95b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65674049, jitterRate=-0.021380409598350525}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:46,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for acfc991c22a20180f5b87d31411aa95b: Running coprocessor pre-open hook at 1676289286235Writing region info on filesystem at 1676289286235Initializing all the Stores at 1676289286236 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289286236Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289286237 (+1 ms)Cleaning up temporary data from old regions at 1676289286242 (+5 ms)Cleaning up detritus from prior splits at 1676289286243 (+1 ms)Running coprocessor post-open hooks at 1676289286246 (+3 ms)Region opened successfully at 1676289286246 2023-02-13 11:54:46,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b., pid=111, masterSystemTime=1676289286232 2023-02-13 11:54:46,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,248 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=acfc991c22a20180f5b87d31411aa95b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,248 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289286248"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289286248"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289286248"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289286248"}]},"ts":"1676289286248"} 2023-02-13 11:54:46,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-02-13 11:54:46,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 in 169 msec 2023-02-13 11:54:46,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-02-13 11:54:46,252 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, ASSIGN in 325 msec 2023-02-13 11:54:46,252 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:54:46,253 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289286252"}]},"ts":"1676289286252"} 2023-02-13 11:54:46,254 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-02-13 11:54:46,283 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:54:46,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 408 msec 2023-02-13 11:54:46,384 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:46,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:46,385 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,385 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,385 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:46,385 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:46,386 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,386 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,396 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,396 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,397 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,397 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,397 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,398 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,399 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,399 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:46,399 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:46,399 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,399 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:46,401 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,407 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,409 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/9a79ddf37ab342c4810d228b53bb535a 2023-02-13 11:54:46,419 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/e0167f243edb4fb9a9f4d7e405a39e14 2023-02-13 11:54:46,423 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/9a79ddf37ab342c4810d228b53bb535a as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a 2023-02-13 11:54:46,427 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:46,428 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/e0167f243edb4fb9a9f4d7e405a39e14 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 2023-02-13 11:54:46,432 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:46,432 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 31ms, sequenceid=6, compaction requested=false 2023-02-13 11:54:46,432 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,433 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:46,433 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,433 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:46,433 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:46,433 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,433 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,433 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,440 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,440 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,441 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,441 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:46,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ec9ecc9[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:46,443 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:46,443 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,449 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,449 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:46,449 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:46,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,450 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,450 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,450 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,457 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,465 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,465 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,465 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,465 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:46,469 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:46,469 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:46,469 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:46,469 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:46,469 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,469 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,480 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,481 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,481 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,482 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,482 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,482 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,482 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,491 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:46,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,490 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,490 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:46,491 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,491 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,492 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,592 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:46,593 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:46,604 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:46,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:46,605 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:46,605 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,605 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,605 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:46,605 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:46,606 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,606 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,615 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,616 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,616 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,616 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,617 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:46,617 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,617 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:46,618 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:46,626 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/1fea3df0debd4bd296217bf254f4ac76 2023-02-13 11:54:46,636 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/64b9f231caa548f6a69102ad93baa350 2023-02-13 11:54:46,639 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/1fea3df0debd4bd296217bf254f4ac76 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 2023-02-13 11:54:46,643 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:46,643 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,644 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/64b9f231caa548f6a69102ad93baa350 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 2023-02-13 11:54:46,647 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:46,648 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 30ms, sequenceid=19, compaction requested=false 2023-02-13 11:54:46,648 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:46,648 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,648 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:46,648 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:46,648 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,649 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,649 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,657 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,665 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,665 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,665 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,666 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,666 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,666 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:46,668 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3ecaf45[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:46,668 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:46,668 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:46,674 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,675 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:46,674 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,675 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,675 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,682 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,682 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,690 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,690 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,690 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:46,694 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:46,694 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:46,694 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:46,694 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:46,694 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,694 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,705 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,706 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,706 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,706 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,707 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,707 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,707 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,708 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,716 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:46,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,716 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,715 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,715 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,716 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,716 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,716 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,816 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:46,817 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:46,828 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:46,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:46,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:46,830 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,830 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,830 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:46,830 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:46,830 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,830 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,840 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,840 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,841 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,841 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,841 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,842 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,842 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,843 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,843 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,842 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:46,843 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,843 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:46,843 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:46,843 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,843 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:46,843 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,849 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,850 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/4b5240cfeb854670a8946801d7ac7092 2023-02-13 11:54:46,860 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/20929c214c614144818110a8cfe80255 2023-02-13 11:54:46,864 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/4b5240cfeb854670a8946801d7ac7092 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 2023-02-13 11:54:46,867 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:46,867 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/20929c214c614144818110a8cfe80255 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 2023-02-13 11:54:46,871 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:46,871 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 28ms, sequenceid=32, compaction requested=true 2023-02-13 11:54:46,871 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:46,871 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:46,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:46,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:46,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,871 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:46,872 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:46,882 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,882 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,883 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,883 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:46,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:46,884 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3b23b36[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:46,884 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:46,884 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,890 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,890 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,890 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,890 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:46,891 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:46,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:46,891 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,899 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,899 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,928 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:46,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,928 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,928 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:46,931 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:46,931 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:46,931 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:46,931 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:46,931 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:46,931 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:46,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:46,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,946 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,946 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:46,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,947 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,948 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,949 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:46,949 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:46,949 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:46,957 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:46,957 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:46,958 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:46,957 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:46,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,958 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:46,958 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:46,958 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:46,959 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:46,959 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,058 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:47,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:47,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:47,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:47,071 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:47,071 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,071 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,072 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:47,072 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:47,072 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,072 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,082 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,082 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,082 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,082 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,082 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,082 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,082 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:47,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,083 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,084 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,084 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:47,084 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:47,084 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,084 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:47,085 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:47,090 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,090 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,091 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,091 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,091 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,091 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,092 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 2023-02-13 11:54:47,102 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 2023-02-13 11:54:47,106 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 2023-02-13 11:54:47,110 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:47,110 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 2023-02-13 11:54:47,114 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:47,114 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 29ms, sequenceid=45, compaction requested=true 2023-02-13 11:54:47,114 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:47,114 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,114 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:47,115 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:47,115 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,115 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,115 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,124 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,124 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,124 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,124 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,124 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,125 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,126 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:47,126 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@d19d9bb[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:47,126 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:47,127 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,132 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,132 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:47,132 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,132 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:47,133 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:47,140 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,140 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,140 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,149 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,150 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,151 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:47,152 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:47,152 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:47,152 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:47,152 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:47,152 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,152 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,157 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,157 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,157 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,158 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,158 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,158 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,158 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,159 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,159 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,159 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,160 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,166 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,167 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,167 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:47,167 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:47,267 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:47,268 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:47,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:47,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:47,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:47,281 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,281 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,281 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:47,281 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:47,281 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,282 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,290 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,290 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,290 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,290 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,291 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,291 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,291 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,292 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,292 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,292 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,292 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,293 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:47,293 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:47,300 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/03c4117b18094e7da514b905bbeb3225 2023-02-13 11:54:47,311 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/f767e1cde62749a393ccb27f3aa73fa6 2023-02-13 11:54:47,314 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/03c4117b18094e7da514b905bbeb3225 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 2023-02-13 11:54:47,315 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,318 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:47,319 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/f767e1cde62749a393ccb27f3aa73fa6 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 2023-02-13 11:54:47,322 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:47,323 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 30ms, sequenceid=58, compaction requested=true 2023-02-13 11:54:47,323 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:47,324 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,324 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,332 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,332 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,333 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,333 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:47,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cc7d375[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:47,334 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:47,334 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,340 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,341 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:47,341 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:47,349 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,349 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,349 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,349 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,357 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:47,361 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:47,361 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:47,361 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:47,361 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:47,361 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,361 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,371 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,371 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,372 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,372 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,372 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,375 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,382 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,382 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:47,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,382 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,383 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,383 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,483 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:47,484 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:47,494 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:47,495 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:47,496 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-02-13 11:54:47,496 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,496 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,496 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-02-13 11:54:47,496 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-02-13 11:54:47,496 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,496 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,505 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,505 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,505 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,505 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:47,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,506 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,506 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,506 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,507 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:47,507 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,507 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. started... 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,507 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:47,514 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/4d06d4f513884f378cdd6ccd7fd0e109 2023-02-13 11:54:47,515 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,515 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,515 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,516 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,516 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,516 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,525 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/8e4572c108384f3dabad86daffd388b5 2023-02-13 11:54:47,532 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/4d06d4f513884f378cdd6ccd7fd0e109 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 2023-02-13 11:54:47,536 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:47,536 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/8e4572c108384f3dabad86daffd388b5 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 2023-02-13 11:54:47,540 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:47,541 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 34ms, sequenceid=71, compaction requested=true 2023-02-13 11:54:47,541 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for acfc991c22a20180f5b87d31411aa95b: 2023-02-13 11:54:47,541 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:47,541 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:47,541 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:47,541 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,541 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-02-13 11:54:47,541 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-02-13 11:54:47,549 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,549 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,549 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,549 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-02-13 11:54:47,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1a37cd2f[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:47,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-02-13 11:54:47,551 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,557 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,558 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:47,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-02-13 11:54:47,558 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:47,565 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,565 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,565 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,600 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-02-13 11:54:47,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,600 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-02-13 11:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,602 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:47,604 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-02-13 11:54:47,604 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-02-13 11:54:47,604 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:47,604 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:47,604 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-02-13 11:54:47,604 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,615 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,615 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:47,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,616 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:47,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,616 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,617 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,618 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:47,618 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-02-13 11:54:47,618 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,619 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:47,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-02-13 11:54:47,630 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:47,630 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:47,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:47,630 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:47,630 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,631 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:47,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:47,631 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-02-13 11:54:47,731 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:47,732 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:47,991 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-02-13 11:54:47,992 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:47,993 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:47,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:47,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:47,994 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:47,995 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:47,995 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:47,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:47,995 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:48,138 ERROR [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid 94b0e9f0-f834-4d10-b805-49b63beb355b) service to localhost.localdomain/127.0.0.1:44261 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:54:48,243 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,243 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:48,244 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:48,244 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:48,245 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:48,246 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-02-13 11:54:48,246 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:48,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:48,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:48,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:48,247 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,248 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,249 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,250 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,251 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,252 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:48,252 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,252 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:48,252 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,252 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,252 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,252 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:48,253 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:48,253 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:48,254 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289318254 2023-02-13 11:54:48,255 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,255 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:48,257 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:48,257 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:48,257 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:48,257 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289348257 2023-02-13 11:54:48,257 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289408257 2023-02-13 11:54:48,258 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:54:48,258 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:48,259 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:48,260 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:48,260 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:48,508 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,508 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:48,508 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:48,508 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:48,509 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:48,509 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:48,509 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:48,509 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:48,509 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:48,510 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:48,512 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-02-13 11:54:48,512 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:48,512 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:48,512 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,513 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:48,514 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,515 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,516 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:48,517 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,517 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,517 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,517 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:48,518 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:48,519 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:48,519 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289318519 2023-02-13 11:54:48,520 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:48,520 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,522 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:48,523 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:48,523 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:48,523 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289348523 2023-02-13 11:54:48,523 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289408523 2023-02-13 11:54:48,523 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:48,523 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:48,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN}] 2023-02-13 11:54:48,525 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:48,525 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,525 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:48,526 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:48,716 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:54:48,773 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,773 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:48,773 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:48,773 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:48,774 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:48,775 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:48,775 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:48,775 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:48,775 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:48,775 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:48,777 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-02-13 11:54:48,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:48,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:48,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:48,777 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,778 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:48,779 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:48,780 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,781 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:48,782 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:48,782 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,782 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:48,782 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,782 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,782 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,782 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:54:48,783 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:48,783 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:48,786 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289318786 2023-02-13 11:54:48,788 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,788 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:48,790 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:48,791 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:48,791 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:48,791 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289348791 2023-02-13 11:54:48,791 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289408791 2023-02-13 11:54:48,791 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:48,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN}] 2023-02-13 11:54:48,792 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:48,795 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:48,795 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:48,796 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289288795"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289288795"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289288795"}]},"ts":"1676289288795"} 2023-02-13 11:54:48,797 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:48,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing acfc991c22a20180f5b87d31411aa95b, disabling compactions & flushes 2023-02-13 11:54:48,949 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. after waiting 0 ms 2023-02-13 11:54:48,949 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:48,950 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing acfc991c22a20180f5b87d31411aa95b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:54:48,959 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/559c7a6204ac4304abcbaa64244144f4 2023-02-13 11:54:48,970 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/8a47dce026e1432bb7f38f8c8a925d6e 2023-02-13 11:54:48,974 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf1/559c7a6204ac4304abcbaa64244144f4 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 2023-02-13 11:54:48,977 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:48,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/.tmp/cf2/8a47dce026e1432bb7f38f8c8a925d6e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e 2023-02-13 11:54:48,981 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:48,982 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 33ms, sequenceid=82, compaction requested=true 2023-02-13 11:54:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:54:48,988 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:48,989 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. 2023-02-13 11:54:48,989 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for acfc991c22a20180f5b87d31411aa95b: Waiting for close lock at 1676289288949Running coprocessor pre-close hooks at 1676289288949Disabling compacts and flushes for region at 1676289288949Disabling writes for close at 1676289288949Obtaining lock to block concurrent updates at 1676289288950 (+1 ms)Preparing flush snapshotting stores in acfc991c22a20180f5b87d31411aa95b at 1676289288950Finished memstore snapshotting testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289288950Flushing stores of testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. at 1676289288950Flushing acfc991c22a20180f5b87d31411aa95b/cf1: creating writer at 1676289288951 (+1 ms)Flushing acfc991c22a20180f5b87d31411aa95b/cf1: appending metadata at 1676289288953 (+2 ms)Flushing acfc991c22a20180f5b87d31411aa95b/cf1: closing flushed file at 1676289288953Flushing acfc991c22a20180f5b87d31411aa95b/cf2: creating writer at 1676289288963 (+10 ms)Flushing acfc991c22a20180f5b87d31411aa95b/cf2: appending metadata at 1676289288965 (+2 ms)Flushing acfc991c22a20180f5b87d31411aa95b/cf2: closing flushed file at 1676289288965Flushing acfc991c22a20180f5b87d31411aa95b/cf1: reopening flushed file at 1676289288974 (+9 ms)Flushing acfc991c22a20180f5b87d31411aa95b/cf2: reopening flushed file at 1676289288978 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for acfc991c22a20180f5b87d31411aa95b in 33ms, sequenceid=82, compaction requested=true at 1676289288982 (+4 ms)Writing region close event to WAL at 1676289288985 (+3 ms)Running coprocessor post-close hooks at 1676289288988 (+3 ms)Closed at 1676289288989 (+1 ms) 2023-02-13 11:54:48,992 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:48,992 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED 2023-02-13 11:54:48,992 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289288992"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289288992"}]},"ts":"1676289288992"} 2023-02-13 11:54:48,994 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-02-13 11:54:48,995 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 in 196 msec 2023-02-13 11:54:48,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-02-13 11:54:48,996 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN in 203 msec 2023-02-13 11:54:48,998 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:48,998 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:48,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:48,998 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:49,042 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:49,042 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:49,043 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:49,045 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:49,046 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,047 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:49,048 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:49,049 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:49,050 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:49,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:49,051 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,051 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,051 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:49,051 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:49,051 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:49,052 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:49,052 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:49,056 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289319056 2023-02-13 11:54:49,058 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,058 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:49,061 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:49,061 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:49,061 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:49,061 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289349061 2023-02-13 11:54:49,061 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289409061 2023-02-13 11:54:49,061 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:49,063 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:49,072 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=acfc991c22a20180f5b87d31411aa95b, threads=14 2023-02-13 11:54:49,073 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,073 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,073 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,073 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,073 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,074 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,074 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,074 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,074 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,074 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,075 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,075 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,075 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,075 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,238 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,249 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,249 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,256 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,258 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,264 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,268 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,268 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,269 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,269 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,270 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,525 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,660 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,660 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,660 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region acfc991c22a20180f5b87d31411aa95b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:54:49,687 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:49,687 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,687 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:49,687 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:49,812 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,812 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:49,812 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:49,812 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:49,813 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:49,813 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:49,813 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:49,813 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:49,813 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:49,814 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:49,815 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-02-13 11:54:49,815 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:49,815 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:49,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:49,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:49,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:49,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:49,816 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,817 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,818 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:49,819 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,820 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:49,821 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,821 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,821 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:49,821 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:49,822 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:49,822 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:49,823 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:49,823 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289319823 2023-02-13 11:54:49,824 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:49,824 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:49,827 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:49,827 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:49,828 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:49,828 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289349828 2023-02-13 11:54:49,828 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289409828 2023-02-13 11:54:49,828 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:49,837 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=acfc991c22a20180f5b87d31411aa95b, threads=14 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,837 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,838 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,838 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,838 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,838 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,838 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,844 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,844 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,844 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,965 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:49,981 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,000 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,004 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,016 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,016 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,028 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,028 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,029 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,029 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,030 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,030 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,031 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,033 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 for region: acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,033 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region acfc991c22a20180f5b87d31411aa95b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:54:50,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:50,064 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:54:50,065 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:54:50,065 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:50,065 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:50,065 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:50,078 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,078 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:50,078 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:50,078 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:50,079 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:50,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:50,079 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:50,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:50,079 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:50,079 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:50,080 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:50,081 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,082 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,083 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,084 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,085 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:50,086 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:50,086 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,087 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,087 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,087 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,087 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:50,087 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:50,088 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:50,088 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289320088 2023-02-13 11:54:50,090 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:50,090 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,092 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:50,092 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:50,092 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:50,092 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289350092 2023-02-13 11:54:50,092 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289410092 2023-02-13 11:54:50,092 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:54:50,094 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:50,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:50,097 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:50,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:50,343 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:50,343 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:50,344 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:50,345 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-02-13 11:54:50,345 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:50,345 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:50,345 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,346 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,347 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:50,348 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,349 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:50,350 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:50,351 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,351 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,351 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,351 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,351 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,351 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:54:50,352 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:50,352 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:50,358 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289320358 2023-02-13 11:54:50,360 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,360 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:50,362 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:50,362 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:50,363 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:50,363 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289350363 2023-02-13 11:54:50,363 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289410363 2023-02-13 11:54:50,363 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:54:50,363 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:50,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289290365"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676289290365"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676289290365"}]},"ts":"1676289290365"} 2023-02-13 11:54:50,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289290365"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289290365"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289290365"}]},"ts":"1676289290365"} 2023-02-13 11:54:50,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289290365"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289290365"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289290365"}]},"ts":"1676289290365"} 2023-02-13 11:54:50,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:50,368 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,368 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:50,368 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:50,613 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,613 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:50,613 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:50,613 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:50,613 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:50,614 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:50,614 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:50,614 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:50,614 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:50,614 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:50,615 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-02-13 11:54:50,616 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:50,616 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:50,616 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,617 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,618 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,618 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:50,618 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,618 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,618 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:50,619 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,620 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,621 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:50,622 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,622 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,622 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,622 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,622 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:50,623 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:50,624 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:50,624 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289320624 2023-02-13 11:54:50,626 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:50,626 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,628 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:50,629 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:50,629 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121, table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,629 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=22b65fca06fd647aa15a7f77bc8fea11, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:50,629 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=bf7a17f0a94181a83e00a80e82436465, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:50,629 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:50,629 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289350629 2023-02-13 11:54:50,629 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289410629 2023-02-13 11:54:50,629 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:50,630 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1676289290630"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1676289290630"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1676289290630"}]},"ts":"1676289290630"} 2023-02-13 11:54:50,630 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289290630"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289290630"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289290630"}]},"ts":"1676289290630"} 2023-02-13 11:54:50,630 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289290630"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289290630"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289290630"}]},"ts":"1676289290630"} 2023-02-13 11:54:50,632 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:50,634 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:50,634 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:50,635 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:50,880 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:50,880 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:50,881 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:50,882 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-02-13 11:54:50,882 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:50,882 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:50,882 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:50,883 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:50,884 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:50,885 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,886 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:50,887 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:50,887 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,887 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,887 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,888 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,888 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:50,888 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-02-13 11:54:50,889 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:50,889 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:50,889 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289320889 2023-02-13 11:54:50,891 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,891 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:50,893 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:50,893 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:50,893 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121, table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:50,894 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=22b65fca06fd647aa15a7f77bc8fea11, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:50,894 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=bf7a17f0a94181a83e00a80e82436465, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:50,894 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:50,894 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289350894 2023-02-13 11:54:50,894 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289410894 2023-02-13 11:54:50,894 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 2 msec 2023-02-13 11:54:50,894 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:50,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN}] 2023-02-13 11:54:50,897 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:50,897 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:50,897 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:50,897 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:51,139 ERROR [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid a211374e-09f0-4304-98b9-bf41d73f5c15) service to localhost.localdomain/127.0.0.1:44261 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:54:51,144 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,144 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:51,144 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:51,144 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:51,145 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:51,145 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:51,145 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:51,145 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:51,145 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:51,146 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:51,147 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-02-13 11:54:51,147 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:51,147 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:51,147 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:51,148 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,149 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,150 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,151 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,152 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:51,153 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,153 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,153 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,153 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,153 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-02-13 11:54:51,154 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:51,155 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:51,155 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289321155 2023-02-13 11:54:51,157 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:51,157 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,159 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:51,159 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:51,159 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121, table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:51,159 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=22b65fca06fd647aa15a7f77bc8fea11, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:51,159 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=bf7a17f0a94181a83e00a80e82436465, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-02-13 11:54:51,159 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:51,159 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289351159 2023-02-13 11:54:51,160 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289411160 2023-02-13 11:54:51,160 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 3 msec 2023-02-13 11:54:51,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN}] 2023-02-13 11:54:51,160 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:51,162 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN 2023-02-13 11:54:51,163 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:54:51,163 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN 2023-02-13 11:54:51,164 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN; state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:54:51,313 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:54:51,315 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=22b65fca06fd647aa15a7f77bc8fea11, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,315 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289291315"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289291315"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289291315"}]},"ts":"1676289291315"} 2023-02-13 11:54:51,317 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:51,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=bf7a17f0a94181a83e00a80e82436465, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289291318"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289291318"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289291318"}]},"ts":"1676289291318"} 2023-02-13 11:54:51,319 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:51,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => bf7a17f0a94181a83e00a80e82436465, NAME => 'testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 11:54:51,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,472 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,473 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1 2023-02-13 11:54:51,473 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1 2023-02-13 11:54:51,473 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf7a17f0a94181a83e00a80e82436465 columnFamilyName cf1 2023-02-13 11:54:51,485 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-top 2023-02-13 11:54:51,489 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-top 2023-02-13 11:54:51,492 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-top 2023-02-13 11:54:51,495 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-top 2023-02-13 11:54:51,498 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-top 2023-02-13 11:54:51,502 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-top 2023-02-13 11:54:51,505 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-top 2023-02-13 11:54:51,505 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(310): Store=bf7a17f0a94181a83e00a80e82436465/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:51,505 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,506 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2 2023-02-13 11:54:51,506 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2 2023-02-13 11:54:51,506 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bf7a17f0a94181a83e00a80e82436465 columnFamilyName cf2 2023-02-13 11:54:51,518 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-top 2023-02-13 11:54:51,522 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-top 2023-02-13 11:54:51,525 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-top 2023-02-13 11:54:51,529 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-top 2023-02-13 11:54:51,533 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-top 2023-02-13 11:54:51,536 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-top 2023-02-13 11:54:51,540 DEBUG [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-top 2023-02-13 11:54:51,540 INFO [StoreOpener-bf7a17f0a94181a83e00a80e82436465-1] regionserver.HStore(310): Store=bf7a17f0a94181a83e00a80e82436465/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:51,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:51,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened bf7a17f0a94181a83e00a80e82436465; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58790842, jitterRate=-0.12394818663597107}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:51,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for bf7a17f0a94181a83e00a80e82436465: Running coprocessor pre-open hook at 1676289291471Writing region info on filesystem at 1676289291471Initializing all the Stores at 1676289291472 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289291472Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289291472Cleaning up temporary data from old regions at 1676289291542 (+70 ms)Cleaning up detritus from prior splits at 1676289291542Running coprocessor post-open hooks at 1676289291544 (+2 ms)Region opened successfully at 1676289291544 2023-02-13 11:54:51,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465., pid=118, masterSystemTime=1676289291468 2023-02-13 11:54:51,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:54:51,546 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:51,546 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:51,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,546 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,546 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,546 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): bf7a17f0a94181a83e00a80e82436465/cf1 is initiating minor compaction (all files) 2023-02-13 11:54:51,546 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): bf7a17f0a94181a83e00a80e82436465/cf2 is initiating minor compaction (all files) 2023-02-13 11:54:51,546 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of bf7a17f0a94181a83e00a80e82436465/cf1 in testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,546 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of bf7a17f0a94181a83e00a80e82436465/cf2 in testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,547 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/.tmp, totalSize=34.9 K 2023-02-13 11:54:51,547 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/.tmp, totalSize=34.9 K 2023-02-13 11:54:51,547 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289286378 2023-02-13 11:54:51,547 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289286378 2023-02-13 11:54:51,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:51,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 22b65fca06fd647aa15a7f77bc8fea11, NAME => 'testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 11:54:51,548 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289286593 2023-02-13 11:54:51,548 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=bf7a17f0a94181a83e00a80e82436465, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,548 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289286593 2023-02-13 11:54:51,548 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289291548"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289291548"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289291548"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289291548"}]},"ts":"1676289291548"} 2023-02-13 11:54:51,548 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:51,548 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289286818 2023-02-13 11:54:51,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:51,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,549 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289287060 2023-02-13 11:54:51,548 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289286818 2023-02-13 11:54:51,549 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289287269 2023-02-13 11:54:51,549 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289287060 2023-02-13 11:54:51,555 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289287484 2023-02-13 11:54:51,555 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289287269 2023-02-13 11:54:51,556 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289287733 2023-02-13 11:54:51,556 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,556 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289287484 2023-02-13 11:54:51,557 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289287733 2023-02-13 11:54:51,558 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1 2023-02-13 11:54:51,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-02-13 11:54:51,558 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1 2023-02-13 11:54:51,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121 in 230 msec 2023-02-13 11:54:51,558 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22b65fca06fd647aa15a7f77bc8fea11 columnFamilyName cf1 2023-02-13 11:54:51,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN in 398 msec 2023-02-13 11:54:51,568 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): bf7a17f0a94181a83e00a80e82436465#cf1#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:51,570 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): bf7a17f0a94181a83e00a80e82436465#cf2#compaction#109 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:51,579 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-bottom 2023-02-13 11:54:51,582 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-bottom 2023-02-13 11:54:51,593 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-bottom 2023-02-13 11:54:51,598 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-bottom 2023-02-13 11:54:51,600 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/.tmp/cf1/226a673b7bde4fd49156078b16d66257 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/226a673b7bde4fd49156078b16d66257 2023-02-13 11:54:51,600 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/.tmp/cf2/d57415976d5048d58985836711b146a9 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/d57415976d5048d58985836711b146a9 2023-02-13 11:54:51,603 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-bottom 2023-02-13 11:54:51,605 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bf7a17f0a94181a83e00a80e82436465/cf1 of bf7a17f0a94181a83e00a80e82436465 into 226a673b7bde4fd49156078b16d66257(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:51,605 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf7a17f0a94181a83e00a80e82436465: 2023-02-13 11:54:51,605 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465., storeName=bf7a17f0a94181a83e00a80e82436465/cf1, priority=9, startTime=1676289291545; duration=0sec 2023-02-13 11:54:51,606 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,606 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in bf7a17f0a94181a83e00a80e82436465/cf2 of bf7a17f0a94181a83e00a80e82436465 into d57415976d5048d58985836711b146a9(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:51,606 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for bf7a17f0a94181a83e00a80e82436465: 2023-02-13 11:54:51,606 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465., storeName=bf7a17f0a94181a83e00a80e82436465/cf2, priority=9, startTime=1676289291545; duration=0sec 2023-02-13 11:54:51,606 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,607 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-bottom 2023-02-13 11:54:51,611 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-bottom 2023-02-13 11:54:51,611 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(310): Store=22b65fca06fd647aa15a7f77bc8fea11/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:51,611 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,612 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2 2023-02-13 11:54:51,612 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2 2023-02-13 11:54:51,612 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 22b65fca06fd647aa15a7f77bc8fea11 columnFamilyName cf2 2023-02-13 11:54:51,626 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-bottom 2023-02-13 11:54:51,629 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-bottom 2023-02-13 11:54:51,633 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-bottom 2023-02-13 11:54:51,636 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-bottom 2023-02-13 11:54:51,639 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-bottom 2023-02-13 11:54:51,642 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-bottom 2023-02-13 11:54:51,645 DEBUG [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-bottom 2023-02-13 11:54:51,645 INFO [StoreOpener-22b65fca06fd647aa15a7f77bc8fea11-1] regionserver.HStore(310): Store=22b65fca06fd647aa15a7f77bc8fea11/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:51,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:51,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:51,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 22b65fca06fd647aa15a7f77bc8fea11; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60833918, jitterRate=-0.09350398182868958}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:51,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 22b65fca06fd647aa15a7f77bc8fea11: Running coprocessor pre-open hook at 1676289291549Writing region info on filesystem at 1676289291549Initializing all the Stores at 1676289291555 (+6 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289291556 (+1 ms)Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289291556Cleaning up temporary data from old regions at 1676289291647 (+91 ms)Cleaning up detritus from prior splits at 1676289291647Running coprocessor post-open hooks at 1676289291649 (+2 ms)Region opened successfully at 1676289291649 2023-02-13 11:54:51,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11., pid=117, masterSystemTime=1676289291468 2023-02-13 11:54:51,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,651 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:51,651 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:51,651 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,651 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,651 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,651 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): 22b65fca06fd647aa15a7f77bc8fea11/cf1 is initiating minor compaction (all files) 2023-02-13 11:54:51,651 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): 22b65fca06fd647aa15a7f77bc8fea11/cf2 is initiating minor compaction (all files) 2023-02-13 11:54:51,651 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 22b65fca06fd647aa15a7f77bc8fea11/cf1 in testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,651 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 22b65fca06fd647aa15a7f77bc8fea11/cf2 in testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,652 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/.tmp, totalSize=34.9 K 2023-02-13 11:54:51,652 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/.tmp, totalSize=34.9 K 2023-02-13 11:54:51,652 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289286378 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289286378 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289286593 2023-02-13 11:54:51,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289286593 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289286818 2023-02-13 11:54:51,653 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=22b65fca06fd647aa15a7f77bc8fea11, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289286818 2023-02-13 11:54:51,653 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289291653"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289291653"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289291653"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289291653"}]},"ts":"1676289291653"} 2023-02-13 11:54:51,653 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289287060 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289287060 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289287269 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289287269 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289287484 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289287484 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289287733 2023-02-13 11:54:51,654 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289287733 2023-02-13 11:54:51,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-02-13 11:54:51,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121 in 337 msec 2023-02-13 11:54:51,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-02-13 11:54:51,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN in 497 msec 2023-02-13 11:54:51,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-02-13 11:54:51,659 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:54:51,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-02-13 11:54:51,660 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,660 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-02-13 11:54:51,660 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:54:51,660 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:54:51,660 ERROR [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-02-13 11:54:51,661 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-02-13 11:54:51,661 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-02-13 11:54:51,661 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-02-13 11:54:51,661 INFO [Listener at localhost.localdomain/45781] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-02-13 11:54:51,661 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-02-13 11:54:51,663 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-02-13 11:54:51,664 DEBUG [Listener at localhost.localdomain/45781] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-02-13 11:54:51,664 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-02-13 11:54:51,664 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-02-13 11:54:51,664 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-02-13 11:54:51,665 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-02-13 11:54:51,665 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,665 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=7e2e3da3f4589c85870b60cbdc7b2c06, daughterA=5272e69e5105d902d94749c8ce874cdd, daughterB=bb81a7325747db4cafb6ad2501d634f0 2023-02-13 11:54:51,665 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,665 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-02-13 11:54:51,666 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,666 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=00e2700a0258d2e58039799b308648de, daughterA=553d82e2c9cc5115f61d7f068d12f829, daughterB=873f0d131aa7d6fc989e0bdac352dc63 2023-02-13 11:54:51,666 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,666 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-02-13 11:54:51,666 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,667 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=5249cf8a85fb696a935f1c38070bb429, daughterA=284fe8700b872f36efe1864a6aa81d69, daughterB=1e22a103db85da411d632256c97c4c4a 2023-02-13 11:54:51,667 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=5249cf8a85fb696a935f1c38070bb429, ASSIGN 2023-02-13 11:54:51,667 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,667 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-02-13 11:54:51,667 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,667 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): 22b65fca06fd647aa15a7f77bc8fea11#cf2#compaction#111 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:51,667 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): 22b65fca06fd647aa15a7f77bc8fea11#cf1#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:51,668 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=0ea26b5dc2d93e295dcd198d6f8d85b4, daughterA=567a13b6e959216f77208d4ff8cf6917, daughterB=57492564cd0c4ed8e8e634afd1ddfc55 2023-02-13 11:54:51,668 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,668 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-02-13 11:54:51,668 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,668 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=1f69a2ae9ee6751a9ecdbbcfc72870d6, daughterA=34ef7c7eb6373dbebfbccd213918b582, daughterB=29381611e213b47c8119c7022001ff40 2023-02-13 11:54:51,669 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,669 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-02-13 11:54:51,669 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,669 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,669 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase16.apache.org,44743,1676289185212, table=testRollbackForSplitTableRegionWithReplica, region=889c72465bac480c11b9d4e2bbc6fb8b is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=0cd26988eceb006a09093a6068229c65, daughterA=96679092a2c44b40f4dbd880da99508b, daughterB=2299f99822c5cb317af3c9583a2b5a3f 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-02-13 11:54:51,670 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN 2023-02-13 11:54:51,672 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,673 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:51,673 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 held the lock before restarting, call acquireLock to restore it. 2023-02-13 11:54:51,673 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,673 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,673 INFO [Listener at localhost.localdomain/45781] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,673 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,673 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure acfc991c22a20180f5b87d31411aa95b, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,673 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,674 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,674 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,674 DEBUG [Listener at localhost.localdomain/45781] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121 didn't hold the lock before restarting, skip acquiring lock. 2023-02-13 11:54:51,674 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 12 msec 2023-02-13 11:54:51,676 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-02-13 11:54:51,677 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121, table=hbase:meta, region=1588230740 2023-02-13 11:54:51,677 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1676289321677 2023-02-13 11:54:51,683 DEBUG [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1519): Joining cluster... 2023-02-13 11:54:51,683 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:51,687 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=e1c6db54e2dd7d1f929d02a4343c29a7, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,44743,1676289185212, regionLocation=jenkins-hbase16.apache.org,44743,1676289185212, openSeqNum=2 2023-02-13 11:54:51,687 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=acfc991c22a20180f5b87d31411aa95b, regionState=CLOSED, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=2 2023-02-13 11:54:51,687 INFO [Listener at localhost.localdomain/45781] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase16.apache.org,43159,1676289185121, table=testRecoveryAndDoubleExecution, region=acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:51,687 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=22b65fca06fd647aa15a7f77bc8fea11, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=86 2023-02-13 11:54:51,687 DEBUG [Listener at localhost.localdomain/45781] assignment.RegionStateStore(147): Load hbase:meta entry region=bf7a17f0a94181a83e00a80e82436465, regionState=OPEN, lastHost=jenkins-hbase16.apache.org,43159,1676289185121, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121, openSeqNum=86 2023-02-13 11:54:51,687 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1531): Number of RegionServers=3 2023-02-13 11:54:51,688 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1676289351687 2023-02-13 11:54:51,688 INFO [Listener at localhost.localdomain/45781] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1676289411688 2023-02-13 11:54:51,688 INFO [Listener at localhost.localdomain/45781] assignment.AssignmentManager(1538): Joined the cluster in 6 msec 2023-02-13 11:54:51,688 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-02-13 11:54:51,694 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=acfc991c22a20180f5b87d31411aa95b, daughterA=22b65fca06fd647aa15a7f77bc8fea11, daughterB=bf7a17f0a94181a83e00a80e82436465 in 3.6960 sec 2023-02-13 11:54:51,698 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/.tmp/cf1/d8b7dc24516b4cc991ff9c828d3a404b as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/d8b7dc24516b4cc991ff9c828d3a404b 2023-02-13 11:54:51,701 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/.tmp/cf2/11875fa3a07e45bc82b598f0f9e4a663 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/11875fa3a07e45bc82b598f0f9e4a663 2023-02-13 11:54:51,702 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 22b65fca06fd647aa15a7f77bc8fea11/cf1 of 22b65fca06fd647aa15a7f77bc8fea11 into d8b7dc24516b4cc991ff9c828d3a404b(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:51,702 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 22b65fca06fd647aa15a7f77bc8fea11: 2023-02-13 11:54:51,703 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11., storeName=22b65fca06fd647aa15a7f77bc8fea11/cf1, priority=9, startTime=1676289291650; duration=0sec 2023-02-13 11:54:51,703 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,704 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 22b65fca06fd647aa15a7f77bc8fea11/cf2 of 22b65fca06fd647aa15a7f77bc8fea11 into 11875fa3a07e45bc82b598f0f9e4a663(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:51,704 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 22b65fca06fd647aa15a7f77bc8fea11: 2023-02-13 11:54:51,705 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11., storeName=22b65fca06fd647aa15a7f77bc8fea11/cf2, priority=9, startTime=1676289291651; duration=0sec 2023-02-13 11:54:51,705 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:51,939 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43159] ipc.CallRunner(144): callId: 911 service: ClientService methodName: Scan size: 154 connection: 188.40.62.62:45010 deadline: 1676289351939, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b. is not online on jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:52,049 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:54:52,057 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 11:54:52,061 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:54:52,071 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:52,072 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-02-13 11:54:52,072 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testRecoveryAndDoubleExecution 2023-02-13 11:54:52,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,075 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 11:54:52,075 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289292075"}]},"ts":"1676289292075"} 2023-02-13 11:54:52,076 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-02-13 11:54:52,115 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-02-13 11:54:52,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, UNASSIGN}] 2023-02-13 11:54:52,117 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, UNASSIGN 2023-02-13 11:54:52,117 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=bf7a17f0a94181a83e00a80e82436465, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:52,117 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289292117"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289292117"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289292117"}]},"ts":"1676289292117"} 2023-02-13 11:54:52,118 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:52,120 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, UNASSIGN 2023-02-13 11:54:52,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=22b65fca06fd647aa15a7f77bc8fea11, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:52,120 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289292120"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289292120"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289292120"}]},"ts":"1676289292120"} 2023-02-13 11:54:52,121 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:52,176 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 11:54:52,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:52,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing bf7a17f0a94181a83e00a80e82436465, disabling compactions & flushes 2023-02-13 11:54:52,270 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:52,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:52,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. after waiting 0 ms 2023-02-13 11:54:52,270 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:52,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-top] to archive 2023-02-13 11:54:52,272 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:52,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,276 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,277 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,278 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,279 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,280 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,282 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-top] to archive 2023-02-13 11:54:52,282 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:52,284 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,285 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,287 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,288 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,289 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,290 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,291 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:54:52,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:52,294 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465. 2023-02-13 11:54:52,294 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for bf7a17f0a94181a83e00a80e82436465: Waiting for close lock at 1676289292270Running coprocessor pre-close hooks at 1676289292270Disabling compacts and flushes for region at 1676289292270Disabling writes for close at 1676289292270Writing region close event to WAL at 1676289292292 (+22 ms)Running coprocessor post-close hooks at 1676289292294 (+2 ms)Closed at 1676289292294 2023-02-13 11:54:52,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:52,296 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 22b65fca06fd647aa15a7f77bc8fea11, disabling compactions & flushes 2023-02-13 11:54:52,297 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. after waiting 0 ms 2023-02-13 11:54:52,297 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:52,297 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=bf7a17f0a94181a83e00a80e82436465, regionState=CLOSED 2023-02-13 11:54:52,297 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289292297"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289292297"}]},"ts":"1676289292297"} 2023-02-13 11:54:52,298 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4-bottom] to archive 2023-02-13 11:54:52,299 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:52,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-02-13 11:54:52,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure bf7a17f0a94181a83e00a80e82436465, server=jenkins-hbase16.apache.org,43159,1676289185121 in 180 msec 2023-02-13 11:54:52,300 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,301 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=bf7a17f0a94181a83e00a80e82436465, UNASSIGN in 184 msec 2023-02-13 11:54:52,301 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,302 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,303 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,304 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,305 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,306 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,308 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e-bottom] to archive 2023-02-13 11:54:52,308 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:52,309 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,310 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,311 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,311 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,312 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,313 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,314 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:54:52,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:52,318 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11. 2023-02-13 11:54:52,318 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 22b65fca06fd647aa15a7f77bc8fea11: Waiting for close lock at 1676289292297Running coprocessor pre-close hooks at 1676289292297Disabling compacts and flushes for region at 1676289292297Disabling writes for close at 1676289292297Writing region close event to WAL at 1676289292315 (+18 ms)Running coprocessor post-close hooks at 1676289292317 (+2 ms)Closed at 1676289292318 (+1 ms) 2023-02-13 11:54:52,320 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:52,320 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=22b65fca06fd647aa15a7f77bc8fea11, regionState=CLOSED 2023-02-13 11:54:52,320 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1676289292320"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289292320"}]},"ts":"1676289292320"} 2023-02-13 11:54:52,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-02-13 11:54:52,322 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 22b65fca06fd647aa15a7f77bc8fea11, server=jenkins-hbase16.apache.org,43159,1676289185121 in 200 msec 2023-02-13 11:54:52,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-02-13 11:54:52,324 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=22b65fca06fd647aa15a7f77bc8fea11, UNASSIGN in 207 msec 2023-02-13 11:54:52,324 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289292324"}]},"ts":"1676289292324"} 2023-02-13 11:54:52,325 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-02-13 11:54:52,332 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-02-13 11:54:52,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 261 msec 2023-02-13 11:54:52,378 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-02-13 11:54:52,378 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-02-13 11:54:52,379 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testRecoveryAndDoubleExecution 2023-02-13 11:54:52,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,382 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 11:54:52,386 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,386 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:52,386 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:52,388 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/recovered.edits] 2023-02-13 11:54:52,388 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/recovered.edits] 2023-02-13 11:54:52,388 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/recovered.edits] 2023-02-13 11:54:52,391 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/.regioninfo to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/.regioninfo 2023-02-13 11:54:52,391 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/.regioninfo to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/.regioninfo 2023-02-13 11:54:52,391 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/03c4117b18094e7da514b905bbeb3225 2023-02-13 11:54:52,392 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/1fea3df0debd4bd296217bf254f4ac76 2023-02-13 11:54:52,393 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,393 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05 2023-02-13 11:54:52,393 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/03c4117b18094e7da514b905bbeb3225.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,394 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,394 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4b5240cfeb854670a8946801d7ac7092 2023-02-13 11:54:52,394 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/1fea3df0debd4bd296217bf254f4ac76.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,395 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,395 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/4d06d4f513884f378cdd6ccd7fd0e109 2023-02-13 11:54:52,396 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/3b333b1c4ddb4c7fa76eba14e34b2d05.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,396 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,396 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/559c7a6204ac4304abcbaa64244144f4 2023-02-13 11:54:52,397 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/4b5240cfeb854670a8946801d7ac7092.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,397 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,397 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf1/9a79ddf37ab342c4810d228b53bb535a 2023-02-13 11:54:52,398 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/4d06d4f513884f378cdd6ccd7fd0e109.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,398 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,399 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/559c7a6204ac4304abcbaa64244144f4.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,399 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,400 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0 2023-02-13 11:54:52,400 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf1/9a79ddf37ab342c4810d228b53bb535a.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,401 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/20929c214c614144818110a8cfe80255 2023-02-13 11:54:52,402 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,402 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/176a5f478dd8408b82c1dfa7fe7ebeb0.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/64b9f231caa548f6a69102ad93baa350 2023-02-13 11:54:52,403 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,403 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/20929c214c614144818110a8cfe80255.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,403 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8a47dce026e1432bb7f38f8c8a925d6e 2023-02-13 11:54:52,404 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,404 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/64b9f231caa548f6a69102ad93baa350.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,405 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/8e4572c108384f3dabad86daffd388b5 2023-02-13 11:54:52,405 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,406 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/8a47dce026e1432bb7f38f8c8a925d6e.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,406 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/e0167f243edb4fb9a9f4d7e405a39e14 2023-02-13 11:54:52,407 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,407 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/8e4572c108384f3dabad86daffd388b5.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,407 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/cf2/f767e1cde62749a393ccb27f3aa73fa6 2023-02-13 11:54:52,408 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,408 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/e0167f243edb4fb9a9f4d7e405a39e14.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,409 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/22b65fca06fd647aa15a7f77bc8fea11/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,410 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/bf7a17f0a94181a83e00a80e82436465/cf2/f767e1cde62749a393ccb27f3aa73fa6.acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,410 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b/recovered.edits/85.seqid 2023-02-13 11:54:52,411 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/acfc991c22a20180f5b87d31411aa95b 2023-02-13 11:54:52,412 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/d8b7dc24516b4cc991ff9c828d3a404b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf1/d8b7dc24516b4cc991ff9c828d3a404b 2023-02-13 11:54:52,412 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/226a673b7bde4fd49156078b16d66257 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf1/226a673b7bde4fd49156078b16d66257 2023-02-13 11:54:52,414 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/11875fa3a07e45bc82b598f0f9e4a663 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/cf2/11875fa3a07e45bc82b598f0f9e4a663 2023-02-13 11:54:52,414 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/d57415976d5048d58985836711b146a9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/cf2/d57415976d5048d58985836711b146a9 2023-02-13 11:54:52,415 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11/recovered.edits/90.seqid 2023-02-13 11:54:52,416 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465/recovered.edits/90.seqid 2023-02-13 11:54:52,416 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/22b65fca06fd647aa15a7f77bc8fea11 2023-02-13 11:54:52,416 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testRecoveryAndDoubleExecution/bf7a17f0a94181a83e00a80e82436465 2023-02-13 11:54:52,416 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-02-13 11:54:52,418 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,419 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-02-13 11:54:52,421 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-02-13 11:54:52,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,422 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-02-13 11:54:52,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289292422"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:52,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289292422"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:52,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289292422"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:52,423 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:54:52,423 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => acfc991c22a20180f5b87d31411aa95b, NAME => 'testRecoveryAndDoubleExecution,,1676289285875.acfc991c22a20180f5b87d31411aa95b.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 22b65fca06fd647aa15a7f77bc8fea11, NAME => 'testRecoveryAndDoubleExecution,,1676289287992.22b65fca06fd647aa15a7f77bc8fea11.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => bf7a17f0a94181a83e00a80e82436465, NAME => 'testRecoveryAndDoubleExecution,41,1676289287992.bf7a17f0a94181a83e00a80e82436465.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 11:54:52,423 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-02-13 11:54:52,423 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289292423"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:52,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-02-13 11:54:52,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-02-13 11:54:52,434 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 54 msec 2023-02-13 11:54:52,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-02-13 11:54:52,483 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-02-13 11:54:52,496 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=378 (was 370) - Thread LEAK? -, OpenFileDescriptor=841 (was 747) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=215 (was 199) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5289 (was 5347) 2023-02-13 11:54:52,507 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=378, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=215, ProcessCount=169, AvailableMemoryMB=5289 2023-02-13 11:54:52,507 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:52,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:54:52,516 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:54:52,517 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:52,518 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:54:52,519 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:54:52,520 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,521 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 empty. 2023-02-13 11:54:52,521 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,522 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 11:54:52,530 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-02-13 11:54:52,531 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7671): creating {ENCODED => 4fb671299f7e919f059c24c0516bba93, NAME => 'testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:54:52,537 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:52,537 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1603): Closing 4fb671299f7e919f059c24c0516bba93, disabling compactions & flushes 2023-02-13 11:54:52,537 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,538 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,538 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. after waiting 0 ms 2023-02-13 11:54:52,538 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,538 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,538 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1557): Region close journal for 4fb671299f7e919f059c24c0516bba93: Waiting for close lock at 1676289292537Disabling compacts and flushes for region at 1676289292537Disabling writes for close at 1676289292538 (+1 ms)Writing region close event to WAL at 1676289292538Closed at 1676289292538 2023-02-13 11:54:52,539 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:54:52,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289292540"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289292540"}]},"ts":"1676289292540"} 2023-02-13 11:54:52,541 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:54:52,542 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:54:52,542 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289292542"}]},"ts":"1676289292542"} 2023-02-13 11:54:52,543 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-02-13 11:54:52,557 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:54:52,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:54:52,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:54:52,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:54:52,558 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:54:52,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, ASSIGN}] 2023-02-13 11:54:52,560 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, ASSIGN 2023-02-13 11:54:52,560 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:54:52,710 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:54:52,711 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=4fb671299f7e919f059c24c0516bba93, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:52,712 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289292711"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289292711"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289292711"}]},"ts":"1676289292711"} 2023-02-13 11:54:52,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 4fb671299f7e919f059c24c0516bba93, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:52,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 4fb671299f7e919f059c24c0516bba93, NAME => 'testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:54:52,871 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:52,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:52,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,871 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,873 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,874 DEBUG [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1 2023-02-13 11:54:52,874 DEBUG [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1 2023-02-13 11:54:52,874 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb671299f7e919f059c24c0516bba93 columnFamilyName cf1 2023-02-13 11:54:52,875 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] regionserver.HStore(310): Store=4fb671299f7e919f059c24c0516bba93/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:52,875 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,876 DEBUG [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2 2023-02-13 11:54:52,876 DEBUG [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2 2023-02-13 11:54:52,877 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb671299f7e919f059c24c0516bba93 columnFamilyName cf2 2023-02-13 11:54:52,877 INFO [StoreOpener-4fb671299f7e919f059c24c0516bba93-1] regionserver.HStore(310): Store=4fb671299f7e919f059c24c0516bba93/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:52,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,878 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,879 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:52,880 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:52,881 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:54:52,881 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 4fb671299f7e919f059c24c0516bba93; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67873041, jitterRate=0.01138712465763092}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:52,882 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 4fb671299f7e919f059c24c0516bba93: Running coprocessor pre-open hook at 1676289292871Writing region info on filesystem at 1676289292871Initializing all the Stores at 1676289292872 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289292872Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289292872Cleaning up temporary data from old regions at 1676289292879 (+7 ms)Cleaning up detritus from prior splits at 1676289292879Running coprocessor post-open hooks at 1676289292882 (+3 ms)Region opened successfully at 1676289292882 2023-02-13 11:54:52,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93., pid=127, masterSystemTime=1676289292865 2023-02-13 11:54:52,883 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:52,884 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=4fb671299f7e919f059c24c0516bba93, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:52,884 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289292884"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289292884"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289292884"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289292884"}]},"ts":"1676289292884"} 2023-02-13 11:54:52,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-02-13 11:54:52,887 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 4fb671299f7e919f059c24c0516bba93, server=jenkins-hbase16.apache.org,44985,1676289185170 in 172 msec 2023-02-13 11:54:52,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-02-13 11:54:52,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, ASSIGN in 329 msec 2023-02-13 11:54:52,889 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:54:52,889 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289292889"}]},"ts":"1676289292889"} 2023-02-13 11:54:52,890 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-02-13 11:54:52,905 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:54:52,907 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 389 msec 2023-02-13 11:54:53,024 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:53,026 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,026 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,026 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,026 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:53,026 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:53,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,032 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,032 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,032 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:53,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,034 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,036 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,036 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,036 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:53,036 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,037 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:54:53,046 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,046 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,046 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,047 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,047 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,047 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,139 ERROR [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid 94b0e9f0-f834-4d10-b805-49b63beb355b) service to localhost.localdomain/127.0.0.1:44261 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:54:53,446 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/4004c3868c354be2b954da97ddbeaf53 2023-02-13 11:54:53,462 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc 2023-02-13 11:54:53,466 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/4004c3868c354be2b954da97ddbeaf53 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53 2023-02-13 11:54:53,469 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:53,470 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc 2023-02-13 11:54:53,473 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:53,474 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 437ms, sequenceid=6, compaction requested=false 2023-02-13 11:54:53,474 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,474 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:53,474 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:53,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:53,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,474 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,507 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,507 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:53,509 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@84e2ed4[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:53,509 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:53,509 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,515 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:53,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:53,516 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:53,523 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,523 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,523 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,523 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:53,526 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:53,526 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:53,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:53,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:53,527 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,527 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,532 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,532 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,532 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,533 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,533 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,533 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,534 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,534 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,535 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,588 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,588 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,589 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:53,588 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,589 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:53,588 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,589 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:53,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,589 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,590 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,690 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:53,691 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:53,701 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:53,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:53,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,703 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:53,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:53,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,703 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,721 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,721 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,721 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,721 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,721 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,722 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,722 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,722 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,723 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,723 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,724 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:53,724 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:53,724 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,724 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:53,724 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:53,732 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,732 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/d29d21215e8a4a6c81cbe76927ab1199 2023-02-13 11:54:53,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,740 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,740 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,746 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/076ae166b79d4027a5d767cfd9088b3f 2023-02-13 11:54:53,750 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/d29d21215e8a4a6c81cbe76927ab1199 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199 2023-02-13 11:54:53,753 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:53,754 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/076ae166b79d4027a5d767cfd9088b3f as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f 2023-02-13 11:54:53,757 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:53,758 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 34ms, sequenceid=19, compaction requested=false 2023-02-13 11:54:53,758 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:53,758 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,758 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:53,758 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:53,758 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,758 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,758 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,765 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,765 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,766 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,766 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:53,768 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4b59fa67[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:53,768 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:53,768 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,773 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,773 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,773 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,774 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,774 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:53,774 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:53,774 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:53,782 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,782 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,782 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,790 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,790 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,790 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,790 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,790 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,790 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,791 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:53,794 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:53,794 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:53,794 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:53,794 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:53,794 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,794 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:53,804 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,804 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,804 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,804 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,805 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,806 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,806 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,807 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,807 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,807 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,815 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,816 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:53,815 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,815 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:53,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:53,816 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,817 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,817 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,917 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:53,917 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:53,927 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:53,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:53,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,929 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:53,929 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,929 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:53,929 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:53,930 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,930 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,940 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,940 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,940 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,940 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:53,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:53,941 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,941 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,941 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,942 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,942 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,942 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:53,943 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:53,943 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,943 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:53,943 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:53,948 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,955 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/36fdb435e49b4369958bde54debf4db0 2023-02-13 11:54:53,969 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/22768c15223a4d0485e1002c1d9d5ba6 2023-02-13 11:54:53,973 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/36fdb435e49b4369958bde54debf4db0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0 2023-02-13 11:54:53,976 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:53,977 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/22768c15223a4d0485e1002c1d9d5ba6 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6 2023-02-13 11:54:53,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,978 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,978 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,981 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:53,981 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 38ms, sequenceid=32, compaction requested=true 2023-02-13 11:54:53,982 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:53,982 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:53,982 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:53,996 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,996 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:53,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:53,997 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,997 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:53,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:53,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:53,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:53,998 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@947c402[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:53,998 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:53,999 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,007 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:54,007 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,007 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,007 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,007 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:54,008 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:54,008 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,008 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,008 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:54,015 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,015 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,015 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,023 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,024 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,023 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,023 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:54,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:54,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,024 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,024 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:54,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:54,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:54,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,026 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:54,027 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:54,027 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:54,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:54,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:54,027 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,027 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:54,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,032 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,033 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:54,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,033 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:54,034 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,035 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,046 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,046 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:54,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:54,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,046 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:54,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,047 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,048 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,147 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:54,148 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:54,158 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:54,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:54,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,160 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:54,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:54,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:54,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,160 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,165 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,165 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,166 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,166 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:54,165 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,166 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,167 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,167 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,167 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:54,168 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,168 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:54,168 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:54,168 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:54,168 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:54,168 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:54,173 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,182 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,182 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:54,182 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,182 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,182 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:54,490 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:54:54,576 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/10291c35c7a546358db103557dfab4e9 2023-02-13 11:54:54,591 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/ab018a6f9e0441d19ddc6821fdf11435 2023-02-13 11:54:54,595 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/10291c35c7a546358db103557dfab4e9 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9 2023-02-13 11:54:54,598 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:54,599 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/ab018a6f9e0441d19ddc6821fdf11435 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435 2023-02-13 11:54:54,602 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:54,603 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 435ms, sequenceid=45, compaction requested=true 2023-02-13 11:54:54,603 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:54,603 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:54,604 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:54,604 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:54,604 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,604 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:54,604 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:54,632 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,632 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:54,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:54,632 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,632 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:54,634 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:54,635 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2dbd3eb0[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:54,635 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:54,635 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:54,641 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:54,641 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:54,648 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,648 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,649 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,657 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,665 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:54,665 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,665 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:54,665 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:54,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:54,669 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:54,669 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:54,669 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:54,669 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:54,669 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,669 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:54,679 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,679 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,679 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,679 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,679 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:54,680 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,680 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,681 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:54,681 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:54,682 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,682 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,683 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,742 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,742 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,742 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:54,742 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,742 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:54,742 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:54,743 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:54,743 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,744 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,844 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:54,845 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:54,855 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:54,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:54,856 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,856 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:54,856 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,856 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:54,856 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:54,857 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,857 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,872 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,872 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:54,872 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:54,872 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:54,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,874 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,874 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,874 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:54,874 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:54,874 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:54,874 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:54,874 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:54,880 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:55,283 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/297c215354b24c42aebe1170012cc958 2023-02-13 11:54:55,295 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/6608eaf0b4464cc99d9cc97d9c43e109 2023-02-13 11:54:55,298 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/297c215354b24c42aebe1170012cc958 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958 2023-02-13 11:54:55,302 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:55,303 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/6608eaf0b4464cc99d9cc97d9c43e109 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109 2023-02-13 11:54:55,307 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:55,308 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 434ms, sequenceid=58, compaction requested=true 2023-02-13 11:54:55,308 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:55,308 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:55,308 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:55,308 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:55,308 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,308 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:55,308 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:55,343 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,343 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:55,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:55,343 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,343 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:55,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:55,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:55,345 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@54fc8d5d[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:55,345 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:55,346 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:55,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,355 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:55,355 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:55,355 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:55,356 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:55,363 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:55,363 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,363 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:55,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,363 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:55,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:55,363 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:55,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:55,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:55,372 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:55,372 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,374 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:55,374 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:55,374 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:55,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:55,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:55,374 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,374 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:55,384 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,384 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,384 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,384 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:55,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:55,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,385 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,386 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,386 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:55,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,387 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,387 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:55,387 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:55,397 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:55,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:55,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:55,397 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:55,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,398 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:55,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:55,398 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,398 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,498 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:55,499 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:55,509 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:55,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:55,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,511 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:55,511 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:55,511 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-02-13 11:54:55,511 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-02-13 11:54:55,511 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,511 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,522 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,522 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,522 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:55,522 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:55,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:55,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:55,523 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-02-13 11:54:55,523 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,523 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:55,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:55,524 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:55,524 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. started... 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:55,524 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:55,524 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:55,555 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:55,932 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/cf66fda7693e44379beb2156cb14885c 2023-02-13 11:54:56,140 ERROR [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid a211374e-09f0-4304-98b9-bf41d73f5c15) service to localhost.localdomain/127.0.0.1:44261 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-02-13 11:54:56,344 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/04644fafba714998bdfe70e463139cf3 2023-02-13 11:54:56,349 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/cf66fda7693e44379beb2156cb14885c as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c 2023-02-13 11:54:56,353 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:56,353 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/04644fafba714998bdfe70e463139cf3 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3 2023-02-13 11:54:56,358 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:54:56,358 INFO [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 834ms, sequenceid=71, compaction requested=true 2023-02-13 11:54:56,359 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for 4fb671299f7e919f059c24c0516bba93: 2023-02-13 11:54:56,359 DEBUG [rs(jenkins-hbase16.apache.org,44985,1676289185170)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:56,359 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:56,359 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:56,359 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,359 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-02-13 11:54:56,359 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-02-13 11:54:56,393 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,393 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:56,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:56,394 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,394 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:56,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:56,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-02-13 11:54:56,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5072cf67[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:56,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-02-13 11:54:56,396 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:56,405 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,405 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:56,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,405 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:56,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-02-13 11:54:56,406 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:56,413 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:56,413 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:56,413 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,413 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:56,413 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:56,414 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:56,414 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:56,422 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-02-13 11:54:56,422 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:56,422 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-02-13 11:54:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase16.apache.org,44985,1676289185170': 2023-02-13 11:54:56,425 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-02-13 11:54:56,425 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-02-13 11:54:56,425 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:56,425 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:56,425 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,425 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:56,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,434 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:56,435 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,435 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,435 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:56,435 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,435 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,436 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,436 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,437 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:56,437 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,437 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:56,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-02-13 11:54:56,447 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:56,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:56,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:56,447 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:56,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,447 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,448 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,448 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-02-13 11:54:56,548 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:56,549 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:56,556 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,557 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb671299f7e919f059c24c0516bba93, daughterA=951bb5f9c495de735fba8e756b64708e, daughterB=a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:56,558 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb671299f7e919f059c24c0516bba93, daughterA=951bb5f9c495de735fba8e756b64708e, daughterB=a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:56,558 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb671299f7e919f059c24c0516bba93, daughterA=951bb5f9c495de735fba8e756b64708e, daughterB=a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:56,558 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb671299f7e919f059c24c0516bba93, daughterA=951bb5f9c495de735fba8e756b64708e, daughterB=a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:56,560 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, UNASSIGN}] 2023-02-13 11:54:56,560 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, UNASSIGN 2023-02-13 11:54:56,561 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=4fb671299f7e919f059c24c0516bba93, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:56,561 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289296561"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289296561"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289296561"}]},"ts":"1676289296561"} 2023-02-13 11:54:56,562 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 4fb671299f7e919f059c24c0516bba93, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:56,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:56,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 4fb671299f7e919f059c24c0516bba93, disabling compactions & flushes 2023-02-13 11:54:56,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:56,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:56,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. after waiting 0 ms 2023-02-13 11:54:56,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:56,714 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 4fb671299f7e919f059c24c0516bba93 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:54:56,724 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/0232ee44c7cc4a2797d66ab9a499405d 2023-02-13 11:54:57,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/7b0d299ce96a48809bece25228eb9443 2023-02-13 11:54:57,138 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf1/0232ee44c7cc4a2797d66ab9a499405d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d 2023-02-13 11:54:57,142 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:57,142 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.tmp/cf2/7b0d299ce96a48809bece25228eb9443 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443 2023-02-13 11:54:57,146 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:54:57,147 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 433ms, sequenceid=82, compaction requested=true 2023-02-13 11:54:57,150 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:54:57,151 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:57,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. 2023-02-13 11:54:57,152 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 4fb671299f7e919f059c24c0516bba93: Waiting for close lock at 1676289296714Running coprocessor pre-close hooks at 1676289296714Disabling compacts and flushes for region at 1676289296714Disabling writes for close at 1676289296714Obtaining lock to block concurrent updates at 1676289296714Preparing flush snapshotting stores in 4fb671299f7e919f059c24c0516bba93 at 1676289296714Finished memstore snapshotting testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289296714Flushing stores of testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. at 1676289296715 (+1 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf1: creating writer at 1676289296716 (+1 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf1: appending metadata at 1676289296718 (+2 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf1: closing flushed file at 1676289296718Flushing 4fb671299f7e919f059c24c0516bba93/cf2: creating writer at 1676289296727 (+9 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf2: appending metadata at 1676289296728 (+1 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf2: closing flushed file at 1676289296728Flushing 4fb671299f7e919f059c24c0516bba93/cf1: reopening flushed file at 1676289297138 (+410 ms)Flushing 4fb671299f7e919f059c24c0516bba93/cf2: reopening flushed file at 1676289297143 (+5 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4fb671299f7e919f059c24c0516bba93 in 433ms, sequenceid=82, compaction requested=true at 1676289297147 (+4 ms)Writing region close event to WAL at 1676289297148 (+1 ms)Running coprocessor post-close hooks at 1676289297151 (+3 ms)Closed at 1676289297152 (+1 ms) 2023-02-13 11:54:57,153 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,153 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=4fb671299f7e919f059c24c0516bba93, regionState=CLOSED 2023-02-13 11:54:57,153 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289297153"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289297153"}]},"ts":"1676289297153"} 2023-02-13 11:54:57,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-02-13 11:54:57,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 4fb671299f7e919f059c24c0516bba93, server=jenkins-hbase16.apache.org,44985,1676289185170 in 592 msec 2023-02-13 11:54:57,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-02-13 11:54:57,157 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb671299f7e919f059c24c0516bba93, UNASSIGN in 597 msec 2023-02-13 11:54:57,167 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=4fb671299f7e919f059c24c0516bba93, threads=14 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,167 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,168 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,172 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,172 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,185 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f, top=false 2023-02-13 11:54:57,185 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3, top=false 2023-02-13 11:54:57,186 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109, top=false 2023-02-13 11:54:57,187 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958, top=false 2023-02-13 11:54:57,187 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53, top=false 2023-02-13 11:54:57,188 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443, top=false 2023-02-13 11:54:57,189 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9, top=false 2023-02-13 11:54:57,190 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d, top=false 2023-02-13 11:54:57,190 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435, top=false 2023-02-13 11:54:57,190 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6, top=false 2023-02-13 11:54:57,190 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc, top=false 2023-02-13 11:54:57,192 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0, top=false 2023-02-13 11:54:57,192 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199, top=false 2023-02-13 11:54:57,194 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c, top=false 2023-02-13 11:54:57,208 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,211 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,216 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,219 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,224 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,228 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,237 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,238 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,238 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,238 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,238 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,238 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/.splits/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9 for child: 951bb5f9c495de735fba8e756b64708e, parent: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,239 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,241 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,241 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,241 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,241 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,242 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199 for region: 4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:57,242 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 4fb671299f7e919f059c24c0516bba93 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-02-13 11:54:57,269 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:54:57,271 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:54:57,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1676289297273"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1676289297273"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1676289297273"}]},"ts":"1676289297273"} 2023-02-13 11:54:57,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289297273"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289297273"}]},"ts":"1676289297273"} 2023-02-13 11:54:57,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297273"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289297273"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289297273"}]},"ts":"1676289297273"} 2023-02-13 11:54:57,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, ASSIGN}] 2023-02-13 11:54:57,278 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, ASSIGN 2023-02-13 11:54:57,278 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:54:57,279 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, ASSIGN 2023-02-13 11:54:57,279 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,44985,1676289185170; forceNewPlan=false, retain=false 2023-02-13 11:54:57,428 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:54:57,430 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=951bb5f9c495de735fba8e756b64708e, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,431 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297430"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289297430"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289297430"}]},"ts":"1676289297430"} 2023-02-13 11:54:57,432 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure 951bb5f9c495de735fba8e756b64708e, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:57,434 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=a4fceaf492919ed57ba27a160a322531, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,435 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297434"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289297434"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289297434"}]},"ts":"1676289297434"} 2023-02-13 11:54:57,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure a4fceaf492919ed57ba27a160a322531, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:57,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:57,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 951bb5f9c495de735fba8e756b64708e, NAME => 'testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.', STARTKEY => '', ENDKEY => '71'} 2023-02-13 11:54:57,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:57,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:57,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,587 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,588 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,589 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1 2023-02-13 11:54:57,589 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1 2023-02-13 11:54:57,590 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 951bb5f9c495de735fba8e756b64708e columnFamilyName cf1 2023-02-13 11:54:57,594 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d 2023-02-13 11:54:57,597 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9 2023-02-13 11:54:57,600 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958 2023-02-13 11:54:57,602 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0 2023-02-13 11:54:57,604 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53 2023-02-13 11:54:57,607 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c 2023-02-13 11:54:57,609 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199 2023-02-13 11:54:57,609 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(310): Store=951bb5f9c495de735fba8e756b64708e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:57,610 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,610 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2 2023-02-13 11:54:57,610 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2 2023-02-13 11:54:57,611 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 951bb5f9c495de735fba8e756b64708e columnFamilyName cf2 2023-02-13 11:54:57,615 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3 2023-02-13 11:54:57,618 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f 2023-02-13 11:54:57,621 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6 2023-02-13 11:54:57,623 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109 2023-02-13 11:54:57,625 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443 2023-02-13 11:54:57,628 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435 2023-02-13 11:54:57,630 DEBUG [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc 2023-02-13 11:54:57,630 INFO [StoreOpener-951bb5f9c495de735fba8e756b64708e-1] regionserver.HStore(310): Store=951bb5f9c495de735fba8e756b64708e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:57,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,633 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:57,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:57,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 951bb5f9c495de735fba8e756b64708e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71678916, jitterRate=0.06809908151626587}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:57,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 951bb5f9c495de735fba8e756b64708e: Running coprocessor pre-open hook at 1676289297587Writing region info on filesystem at 1676289297587Initializing all the Stores at 1676289297588 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289297588Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289297588Cleaning up temporary data from old regions at 1676289297632 (+44 ms)Cleaning up detritus from prior splits at 1676289297632Running coprocessor post-open hooks at 1676289297634 (+2 ms)Region opened successfully at 1676289297634 2023-02-13 11:54:57,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e., pid=133, masterSystemTime=1676289297584 2023-02-13 11:54:57,635 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:54:57,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1912): 951bb5f9c495de735fba8e756b64708e/cf2 is initiating minor compaction (all files) 2023-02-13 11:54:57,636 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1912): 951bb5f9c495de735fba8e756b64708e/cf1 is initiating minor compaction (all files) 2023-02-13 11:54:57,636 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 951bb5f9c495de735fba8e756b64708e/cf2 in testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:57,636 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 951bb5f9c495de735fba8e756b64708e/cf1 in testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:57,637 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/.tmp, totalSize=34.9 K 2023-02-13 11:54:57,637 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/.tmp, totalSize=34.9 K 2023-02-13 11:54:57,637 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289293020 2023-02-13 11:54:57,637 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289293020 2023-02-13 11:54:57,637 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289293691 2023-02-13 11:54:57,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:57,637 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289293691 2023-02-13 11:54:57,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:57,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289293918 2023-02-13 11:54:57,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => a4fceaf492919ed57ba27a160a322531, NAME => 'testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.', STARTKEY => '71', ENDKEY => ''} 2023-02-13 11:54:57,638 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=951bb5f9c495de735fba8e756b64708e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289293918 2023-02-13 11:54:57,638 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297638"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289297638"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289297638"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289297638"}]},"ts":"1676289297638"} 2023-02-13 11:54:57,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289294149 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289294149 2023-02-13 11:54:57,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289294846 2023-02-13 11:54:57,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289294846 2023-02-13 11:54:57,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,638 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289295499 2023-02-13 11:54:57,639 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289295499 2023-02-13 11:54:57,639 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289296549 2023-02-13 11:54:57,639 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289296549 2023-02-13 11:54:57,641 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,643 DEBUG [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf1 2023-02-13 11:54:57,643 DEBUG [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf1 2023-02-13 11:54:57,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-02-13 11:54:57,643 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure 951bb5f9c495de735fba8e756b64708e, server=jenkins-hbase16.apache.org,44985,1676289185170 in 207 msec 2023-02-13 11:54:57,643 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4fceaf492919ed57ba27a160a322531 columnFamilyName cf1 2023-02-13 11:54:57,644 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] regionserver.HStore(310): Store=a4fceaf492919ed57ba27a160a322531/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:57,644 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, ASSIGN in 366 msec 2023-02-13 11:54:57,644 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,646 DEBUG [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf2 2023-02-13 11:54:57,646 DEBUG [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf2 2023-02-13 11:54:57,646 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a4fceaf492919ed57ba27a160a322531 columnFamilyName cf2 2023-02-13 11:54:57,647 INFO [StoreOpener-a4fceaf492919ed57ba27a160a322531-1] regionserver.HStore(310): Store=a4fceaf492919ed57ba27a160a322531/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:57,647 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,649 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:57,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:57,650 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened a4fceaf492919ed57ba27a160a322531; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71055129, jitterRate=0.05880393087863922}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:57,650 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for a4fceaf492919ed57ba27a160a322531: Running coprocessor pre-open hook at 1676289297638Writing region info on filesystem at 1676289297638Initializing all the Stores at 1676289297639 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289297639Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289297639Cleaning up temporary data from old regions at 1676289297648 (+9 ms)Cleaning up detritus from prior splits at 1676289297649 (+1 ms)Running coprocessor post-open hooks at 1676289297650 (+1 ms)Region opened successfully at 1676289297650 2023-02-13 11:54:57,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531., pid=134, masterSystemTime=1676289297584 2023-02-13 11:54:57,653 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:57,653 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:57,653 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=a4fceaf492919ed57ba27a160a322531, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,654 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297653"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289297653"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289297653"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289297653"}]},"ts":"1676289297653"} 2023-02-13 11:54:57,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-02-13 11:54:57,657 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure a4fceaf492919ed57ba27a160a322531, server=jenkins-hbase16.apache.org,44985,1676289185170 in 219 msec 2023-02-13 11:54:57,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-02-13 11:54:57,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, ASSIGN in 380 msec 2023-02-13 11:54:57,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb671299f7e919f059c24c0516bba93, daughterA=951bb5f9c495de735fba8e756b64708e, daughterB=a4fceaf492919ed57ba27a160a322531 in 1.1030 sec 2023-02-13 11:54:57,678 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] throttle.PressureAwareThroughputController(145): 951bb5f9c495de735fba8e756b64708e#cf2#compaction#127 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:57,678 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] throttle.PressureAwareThroughputController(145): 951bb5f9c495de735fba8e756b64708e#cf1#compaction#126 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:54:57,696 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/.tmp/cf2/cf2666c2abe0473ca91e7ae6f3bf719f as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/cf2666c2abe0473ca91e7ae6f3bf719f 2023-02-13 11:54:57,697 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/.tmp/cf1/8868bd233bcf406396461a5301c0cd7e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/8868bd233bcf406396461a5301c0cd7e 2023-02-13 11:54:57,700 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 951bb5f9c495de735fba8e756b64708e/cf2 of 951bb5f9c495de735fba8e756b64708e into cf2666c2abe0473ca91e7ae6f3bf719f(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:57,700 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 951bb5f9c495de735fba8e756b64708e: 2023-02-13 11:54:57,700 INFO [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e., storeName=951bb5f9c495de735fba8e756b64708e/cf2, priority=9, startTime=1676289297636; duration=0sec 2023-02-13 11:54:57,700 DEBUG [RS:1;jenkins-hbase16:44985-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:57,701 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 951bb5f9c495de735fba8e756b64708e/cf1 of 951bb5f9c495de735fba8e756b64708e into 8868bd233bcf406396461a5301c0cd7e(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:54:57,701 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 951bb5f9c495de735fba8e756b64708e: 2023-02-13 11:54:57,701 INFO [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e., storeName=951bb5f9c495de735fba8e756b64708e/cf1, priority=9, startTime=1676289297635; duration=0sec 2023-02-13 11:54:57,702 DEBUG [RS:1;jenkins-hbase16:44985-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:54:57,808 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44985] ipc.CallRunner(144): callId: 1042 service: ClientService methodName: Scan size: 157 connection: 188.40.62.62:44498 deadline: 1676289357808, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93. is not online on jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,919 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-02-13 11:54:57,924 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:57,925 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-02-13 11:54:57,925 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegionEmptyDaughter 2023-02-13 11:54:57,926 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:57,928 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 11:54:57,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289297928"}]},"ts":"1676289297928"} 2023-02-13 11:54:57,929 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-02-13 11:54:57,959 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-02-13 11:54:57,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, UNASSIGN}] 2023-02-13 11:54:57,961 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, UNASSIGN 2023-02-13 11:54:57,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=a4fceaf492919ed57ba27a160a322531, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297962"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289297962"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289297962"}]},"ts":"1676289297962"} 2023-02-13 11:54:57,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure a4fceaf492919ed57ba27a160a322531, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:57,965 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, UNASSIGN 2023-02-13 11:54:57,965 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=951bb5f9c495de735fba8e756b64708e, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:57,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289297965"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289297965"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289297965"}]},"ts":"1676289297965"} 2023-02-13 11:54:57,967 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 951bb5f9c495de735fba8e756b64708e, server=jenkins-hbase16.apache.org,44985,1676289185170}] 2023-02-13 11:54:58,030 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 11:54:58,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:58,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 951bb5f9c495de735fba8e756b64708e, disabling compactions & flushes 2023-02-13 11:54:58,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:58,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:58,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. after waiting 0 ms 2023-02-13 11:54:58,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:58,116 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d] to archive 2023-02-13 11:54:58,116 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:58,118 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-4004c3868c354be2b954da97ddbeaf53 2023-02-13 11:54:58,119 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-d29d21215e8a4a6c81cbe76927ab1199 2023-02-13 11:54:58,120 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-36fdb435e49b4369958bde54debf4db0 2023-02-13 11:54:58,121 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-10291c35c7a546358db103557dfab4e9 2023-02-13 11:54:58,122 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-297c215354b24c42aebe1170012cc958 2023-02-13 11:54:58,123 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-cf66fda7693e44379beb2156cb14885c 2023-02-13 11:54:58,124 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-0232ee44c7cc4a2797d66ab9a499405d 2023-02-13 11:54:58,125 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443] to archive 2023-02-13 11:54:58,126 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:54:58,127 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-b34702c57a9c47c1b7d4f3aa83dc42cc 2023-02-13 11:54:58,128 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-076ae166b79d4027a5d767cfd9088b3f 2023-02-13 11:54:58,129 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-22768c15223a4d0485e1002c1d9d5ba6 2023-02-13 11:54:58,130 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-ab018a6f9e0441d19ddc6821fdf11435 2023-02-13 11:54:58,131 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-6608eaf0b4464cc99d9cc97d9c43e109 2023-02-13 11:54:58,132 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-04644fafba714998bdfe70e463139cf3 2023-02-13 11:54:58,133 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/testSplitTableRegionEmptyDaughter=4fb671299f7e919f059c24c0516bba93-7b0d299ce96a48809bece25228eb9443 2023-02-13 11:54:58,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:54:58,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:58,137 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e. 2023-02-13 11:54:58,137 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 951bb5f9c495de735fba8e756b64708e: Waiting for close lock at 1676289298115Running coprocessor pre-close hooks at 1676289298115Disabling compacts and flushes for region at 1676289298115Disabling writes for close at 1676289298115Writing region close event to WAL at 1676289298134 (+19 ms)Running coprocessor post-close hooks at 1676289298137 (+3 ms)Closed at 1676289298137 2023-02-13 11:54:58,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:58,138 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:58,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing a4fceaf492919ed57ba27a160a322531, disabling compactions & flushes 2023-02-13 11:54:58,139 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:58,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:58,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. after waiting 0 ms 2023-02-13 11:54:58,139 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:58,139 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=951bb5f9c495de735fba8e756b64708e, regionState=CLOSED 2023-02-13 11:54:58,140 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289298139"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289298139"}]},"ts":"1676289298139"} 2023-02-13 11:54:58,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-02-13 11:54:58,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 951bb5f9c495de735fba8e756b64708e, server=jenkins-hbase16.apache.org,44985,1676289185170 in 174 msec 2023-02-13 11:54:58,143 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-02-13 11:54:58,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:54:58,144 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531. 2023-02-13 11:54:58,144 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for a4fceaf492919ed57ba27a160a322531: Waiting for close lock at 1676289298138Running coprocessor pre-close hooks at 1676289298138Disabling compacts and flushes for region at 1676289298138Disabling writes for close at 1676289298139 (+1 ms)Writing region close event to WAL at 1676289298140 (+1 ms)Running coprocessor post-close hooks at 1676289298144 (+4 ms)Closed at 1676289298144 2023-02-13 11:54:58,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=951bb5f9c495de735fba8e756b64708e, UNASSIGN in 183 msec 2023-02-13 11:54:58,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:58,145 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=a4fceaf492919ed57ba27a160a322531, regionState=CLOSED 2023-02-13 11:54:58,146 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1676289298145"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289298145"}]},"ts":"1676289298145"} 2023-02-13 11:54:58,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-02-13 11:54:58,148 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure a4fceaf492919ed57ba27a160a322531, server=jenkins-hbase16.apache.org,44985,1676289185170 in 184 msec 2023-02-13 11:54:58,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-02-13 11:54:58,149 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=a4fceaf492919ed57ba27a160a322531, UNASSIGN in 188 msec 2023-02-13 11:54:58,149 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289298149"}]},"ts":"1676289298149"} 2023-02-13 11:54:58,150 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-02-13 11:54:58,159 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-02-13 11:54:58,161 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 235 msec 2023-02-13 11:54:58,231 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-02-13 11:54:58,231 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-02-13 11:54:58,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,235 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,235 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,235 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 11:54:58,239 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:58,239 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:58,239 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:58,240 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/recovered.edits] 2023-02-13 11:54:58,240 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/recovered.edits] 2023-02-13 11:54:58,241 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/recovered.edits] 2023-02-13 11:54:58,243 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/8868bd233bcf406396461a5301c0cd7e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf1/8868bd233bcf406396461a5301c0cd7e 2023-02-13 11:54:58,243 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/0232ee44c7cc4a2797d66ab9a499405d 2023-02-13 11:54:58,244 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/10291c35c7a546358db103557dfab4e9 2023-02-13 11:54:58,245 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/recovered.edits/88.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531/recovered.edits/88.seqid 2023-02-13 11:54:58,245 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/cf2666c2abe0473ca91e7ae6f3bf719f to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/cf2/cf2666c2abe0473ca91e7ae6f3bf719f 2023-02-13 11:54:58,245 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/297c215354b24c42aebe1170012cc958 2023-02-13 11:54:58,245 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/a4fceaf492919ed57ba27a160a322531 2023-02-13 11:54:58,246 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/36fdb435e49b4369958bde54debf4db0 2023-02-13 11:54:58,247 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/4004c3868c354be2b954da97ddbeaf53 2023-02-13 11:54:58,247 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e/recovered.edits/90.seqid 2023-02-13 11:54:58,247 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/951bb5f9c495de735fba8e756b64708e 2023-02-13 11:54:58,247 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/cf66fda7693e44379beb2156cb14885c 2023-02-13 11:54:58,248 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf1/d29d21215e8a4a6c81cbe76927ab1199 2023-02-13 11:54:58,250 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/04644fafba714998bdfe70e463139cf3 2023-02-13 11:54:58,251 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/076ae166b79d4027a5d767cfd9088b3f 2023-02-13 11:54:58,253 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/22768c15223a4d0485e1002c1d9d5ba6 2023-02-13 11:54:58,254 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/6608eaf0b4464cc99d9cc97d9c43e109 2023-02-13 11:54:58,255 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/7b0d299ce96a48809bece25228eb9443 2023-02-13 11:54:58,256 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/ab018a6f9e0441d19ddc6821fdf11435 2023-02-13 11:54:58,257 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/cf2/b34702c57a9c47c1b7d4f3aa83dc42cc 2023-02-13 11:54:58,259 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93/recovered.edits/85.seqid 2023-02-13 11:54:58,260 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb671299f7e919f059c24c0516bba93 2023-02-13 11:54:58,260 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-02-13 11:54:58,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,263 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-02-13 11:54:58,265 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-02-13 11:54:58,266 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,266 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-02-13 11:54:58,266 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289298266"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:58,266 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289298266"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:58,266 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289298266"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:58,268 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:54:58,268 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4fb671299f7e919f059c24c0516bba93, NAME => 'testSplitTableRegionEmptyDaughter,,1676289292516.4fb671299f7e919f059c24c0516bba93.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 951bb5f9c495de735fba8e756b64708e, NAME => 'testSplitTableRegionEmptyDaughter,,1676289296556.951bb5f9c495de735fba8e756b64708e.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => a4fceaf492919ed57ba27a160a322531, NAME => 'testSplitTableRegionEmptyDaughter,71,1676289296556.a4fceaf492919ed57ba27a160a322531.', STARTKEY => '71', ENDKEY => ''}] 2023-02-13 11:54:58,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-02-13 11:54:58,268 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289298268"}]},"ts":"9223372036854775807"} 2023-02-13 11:54:58,270 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-02-13 11:54:58,280 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-02-13 11:54:58,281 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 49 msec 2023-02-13 11:54:58,336 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-02-13 11:54:58,337 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-02-13 11:54:58,351 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 378), OpenFileDescriptor=841 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=229 (was 215) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=5298 (was 5289) - AvailableMemoryMB LEAK? - 2023-02-13 11:54:58,365 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=229, ProcessCount=169, AvailableMemoryMB=5298 2023-02-13 11:54:58,365 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:54:58,376 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-02-13 11:54:58,377 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(492): Client=jenkins//188.40.62.62 set balanceSwitch=false 2023-02-13 11:54:58,378 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-02-13 11:54:58,380 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-02-13 11:54:58,380 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-02-13 11:54:58,381 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,382 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd empty. 2023-02-13 11:54:58,382 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,382 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 11:54:58,391 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-02-13 11:54:58,392 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7671): creating {ENCODED => e59db06e6e7d32a7f703cd50dccda9fd, NAME => 'testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1603): Closing e59db06e6e7d32a7f703cd50dccda9fd, disabling compactions & flushes 2023-02-13 11:54:58,398 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. after waiting 0 ms 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,398 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1557): Region close journal for e59db06e6e7d32a7f703cd50dccda9fd: Waiting for close lock at 1676289298398Disabling compacts and flushes for region at 1676289298398Disabling writes for close at 1676289298398Writing region close event to WAL at 1676289298398Closed at 1676289298398 2023-02-13 11:54:58,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-02-13 11:54:58,401 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289298401"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289298401"}]},"ts":"1676289298401"} 2023-02-13 11:54:58,402 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-02-13 11:54:58,403 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-02-13 11:54:58,403 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289298403"}]},"ts":"1676289298403"} 2023-02-13 11:54:58,404 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-02-13 11:54:58,422 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase16.apache.org=0} racks are {/default-rack=0} 2023-02-13 11:54:58,423 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-02-13 11:54:58,423 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-02-13 11:54:58,423 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-02-13 11:54:58,423 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-02-13 11:54:58,423 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, ASSIGN}] 2023-02-13 11:54:58,424 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, ASSIGN 2023-02-13 11:54:58,425 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, ASSIGN; state=OFFLINE, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:54:58,575 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-02-13 11:54:58,576 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e59db06e6e7d32a7f703cd50dccda9fd, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,576 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289298576"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289298576"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289298576"}]},"ts":"1676289298576"} 2023-02-13 11:54:58,578 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure e59db06e6e7d32a7f703cd50dccda9fd, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:54:58,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => e59db06e6e7d32a7f703cd50dccda9fd, NAME => 'testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.', STARTKEY => '', ENDKEY => ''} 2023-02-13 11:54:58,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:54:58,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:54:58,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,734 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,735 DEBUG [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1 2023-02-13 11:54:58,736 DEBUG [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1 2023-02-13 11:54:58,736 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e59db06e6e7d32a7f703cd50dccda9fd columnFamilyName cf1 2023-02-13 11:54:58,736 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] regionserver.HStore(310): Store=e59db06e6e7d32a7f703cd50dccda9fd/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:58,736 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,737 DEBUG [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2 2023-02-13 11:54:58,737 DEBUG [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2 2023-02-13 11:54:58,738 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e59db06e6e7d32a7f703cd50dccda9fd columnFamilyName cf2 2023-02-13 11:54:58,738 INFO [StoreOpener-e59db06e6e7d32a7f703cd50dccda9fd-1] regionserver.HStore(310): Store=e59db06e6e7d32a7f703cd50dccda9fd/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:54:58,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:54:58,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:54:58,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-02-13 11:54:58,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened e59db06e6e7d32a7f703cd50dccda9fd; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=61708900, jitterRate=-0.08046573400497437}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:54:58,742 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for e59db06e6e7d32a7f703cd50dccda9fd: Running coprocessor pre-open hook at 1676289298733Writing region info on filesystem at 1676289298733Initializing all the Stores at 1676289298734 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289298734Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289298734Cleaning up temporary data from old regions at 1676289298739 (+5 ms)Cleaning up detritus from prior splits at 1676289298740 (+1 ms)Running coprocessor post-open hooks at 1676289298742 (+2 ms)Region opened successfully at 1676289298742 2023-02-13 11:54:58,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd., pid=143, masterSystemTime=1676289298729 2023-02-13 11:54:58,744 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,744 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e59db06e6e7d32a7f703cd50dccda9fd, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,745 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289298744"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289298744"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289298744"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289298744"}]},"ts":"1676289298744"} 2023-02-13 11:54:58,746 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-02-13 11:54:58,747 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure e59db06e6e7d32a7f703cd50dccda9fd, server=jenkins-hbase16.apache.org,43159,1676289185121 in 167 msec 2023-02-13 11:54:58,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-02-13 11:54:58,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, ASSIGN in 324 msec 2023-02-13 11:54:58,748 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-02-13 11:54:58,748 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289298748"}]},"ts":"1676289298748"} 2023-02-13 11:54:58,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-02-13 11:54:58,760 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-02-13 11:54:58,761 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 383 msec 2023-02-13 11:54:58,885 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:58,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:54:58,887 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:54:58,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:58,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:54:58,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:54:58,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:58,887 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,897 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:58,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:58,897 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:58,897 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,897 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:58,898 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:58,898 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:58,898 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:54:58,898 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:58,899 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:58,899 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:58,899 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:58,899 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=120 B heapSize=928 B 2023-02-13 11:54:58,909 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,910 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/ba8acff1939a4a0c9f24628e137de6ed 2023-02-13 11:54:58,918 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,918 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,918 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:58,918 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,918 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:58,921 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/f072d7aaba844c50ab63dd18af27a43e 2023-02-13 11:54:58,924 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/ba8acff1939a4a0c9f24628e137de6ed as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed 2023-02-13 11:54:58,928 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:58,928 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/f072d7aaba844c50ab63dd18af27a43e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e 2023-02-13 11:54:58,931 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:54:58,932 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 33ms, sequenceid=6, compaction requested=false 2023-02-13 11:54:58,932 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-02-13 11:54:58,933 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:54:58,933 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:58,933 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:58,933 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:58,933 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,933 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:58,933 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:58,943 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:58,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:58,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:58,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:58,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:58,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:54:58,945 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:54:58,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@67a276bf[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:58,945 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:58,977 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:58,977 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,978 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:58,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:58,978 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:58,993 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:58,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,993 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:58,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:58,993 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,001 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,001 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,001 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,001 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,001 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,002 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,002 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:59,005 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:54:59,005 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:54:59,005 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:59,005 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:59,005 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:54:59,005 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:59,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,013 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,014 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,014 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,014 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,015 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,015 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,016 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,017 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,026 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,026 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:54:59,027 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,026 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,026 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,027 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,028 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,028 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,028 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,127 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:59,128 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:59,137 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:59,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:59,139 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:54:59,139 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:54:59,139 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,139 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:54:59,139 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:54:59,139 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,139 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,146 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,147 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,146 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,146 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,147 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,147 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,148 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,148 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:59,149 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,149 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:54:59,149 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:59,149 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:59,155 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,163 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,163 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,163 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,164 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,164 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,165 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/1f389b847f3e440894fc7ec4afeca0e3 2023-02-13 11:54:59,176 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/404da48346b848329591df2eeb214aa8 2023-02-13 11:54:59,179 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/1f389b847f3e440894fc7ec4afeca0e3 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3 2023-02-13 11:54:59,182 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:59,183 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/404da48346b848329591df2eeb214aa8 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8 2023-02-13 11:54:59,186 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8, entries=10, sequenceid=19, filesize=5.0 K 2023-02-13 11:54:59,187 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 38ms, sequenceid=19, compaction requested=false 2023-02-13 11:54:59,188 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:54:59,188 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,188 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:59,188 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:59,188 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,188 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,188 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,197 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,197 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,197 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:54:59,199 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@232e9c45[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:59,199 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:54:59,199 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,205 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,205 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:59,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:59,205 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,205 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,206 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,206 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:59,206 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,206 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:59,213 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,213 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,213 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,222 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,222 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:59,225 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:54:59,225 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:54:59,225 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:59,225 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:59,225 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:54:59,226 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,263 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,264 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,264 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,264 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,265 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,266 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,266 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,298 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,298 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:54:59,299 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:59,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:59,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,298 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:59,299 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,299 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,300 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,399 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:59,400 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:59,409 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:59,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:59,410 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:54:59,410 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:54:59,410 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,411 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:54:59,411 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:54:59,411 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,411 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,422 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,422 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,422 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,422 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,422 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,423 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,423 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,423 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,423 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,423 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:59,424 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,424 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,424 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:54:59,424 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:59,430 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,431 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/16064054a79449ce9ed8a9bf31d83e3b 2023-02-13 11:54:59,438 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,438 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,438 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,438 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,438 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,441 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/261bb0f83b914a7291fe35493b0b9bf6 2023-02-13 11:54:59,445 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/16064054a79449ce9ed8a9bf31d83e3b as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b 2023-02-13 11:54:59,449 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:59,450 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/261bb0f83b914a7291fe35493b0b9bf6 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6 2023-02-13 11:54:59,453 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6, entries=10, sequenceid=32, filesize=5.0 K 2023-02-13 11:54:59,454 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 30ms, sequenceid=32, compaction requested=true 2023-02-13 11:54:59,454 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:54:59,454 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,454 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:59,454 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:59,454 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,454 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,454 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,463 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,463 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,464 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,464 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:54:59,466 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@13086b18[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:59,466 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:54:59,466 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,472 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:59,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:59,472 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:59,480 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,480 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,480 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,480 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,488 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,488 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,488 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,488 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,488 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,489 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:59,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:54:59,491 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:54:59,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:59,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:59,491 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:54:59,491 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,501 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,502 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,501 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,501 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,502 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,502 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,503 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,504 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,504 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,513 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,514 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,513 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:54:59,514 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:59,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,513 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,513 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,514 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:59,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,514 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,514 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,514 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:59,514 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,515 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,615 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:59,616 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:59,625 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:59,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:59,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:54:59,626 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:54:59,626 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,626 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:54:59,626 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:54:59,626 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,626 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,634 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,634 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,635 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,635 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,635 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,636 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,637 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,636 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,637 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,637 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,637 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,637 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,638 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:59,638 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:59,638 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,638 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:54:59,638 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:59,646 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 2023-02-13 11:54:59,646 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,655 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,655 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,655 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,655 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,655 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,656 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/4d525582989e498d9f8a6b0fe3692f92 2023-02-13 11:54:59,659 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 2023-02-13 11:54:59,662 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:59,663 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/4d525582989e498d9f8a6b0fe3692f92 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92 2023-02-13 11:54:59,666 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92, entries=10, sequenceid=45, filesize=5.0 K 2023-02-13 11:54:59,667 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 29ms, sequenceid=45, compaction requested=true 2023-02-13 11:54:59,667 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:54:59,667 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,667 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:59,667 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:59,667 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,667 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,667 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,676 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,676 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,677 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,677 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:54:59,678 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:54:59,678 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55c787e9[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:59,678 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,688 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,688 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,688 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,689 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,689 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:59,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,689 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:59,697 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,697 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,697 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,751 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,751 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:59,755 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:54:59,755 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:54:59,755 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:59,755 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:59,755 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:54:59,755 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,773 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,773 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,773 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,774 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,774 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,775 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,775 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,775 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,784 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,784 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,785 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,784 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,784 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:54:59,785 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:54:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,785 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,785 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,785 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:54:59,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,786 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,786 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,786 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,886 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:54:59,887 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:54:59,896 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:54:59,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 11:54:59,897 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:54:59,898 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:54:59,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:54:59,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:54:59,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,898 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,905 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,905 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,905 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,905 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:54:59,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,906 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,906 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:54:59,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:54:59,906 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:54:59,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,907 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,907 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:54:59,907 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:54:59,907 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,908 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:54:59,908 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:54:59,915 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 2023-02-13 11:54:59,918 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,925 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/5c7c89c6ff724ba2a67af5d0da0add1e 2023-02-13 11:54:59,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,926 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,926 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,926 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,928 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 2023-02-13 11:54:59,931 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:59,931 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/5c7c89c6ff724ba2a67af5d0da0add1e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e 2023-02-13 11:54:59,934 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e, entries=10, sequenceid=58, filesize=5.0 K 2023-02-13 11:54:59,935 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 27ms, sequenceid=58, compaction requested=true 2023-02-13 11:54:59,935 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:54:59,935 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:54:59,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:54:59,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:54:59,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:54:59,935 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:54:59,943 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,943 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:54:59,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:54:59,945 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@21241460[Count = 0] remaining members to acquire global barrier 2023-02-13 11:54:59,945 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:54:59,945 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:54:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:54:59,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:54:59,955 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,956 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,956 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:54:59,956 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:54:59,956 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:54:59,956 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:54:59,963 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,963 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,963 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,972 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,972 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,972 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:54:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:54:59,980 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:54:59,980 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,980 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,981 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:54:59,983 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:54:59,983 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:54:59,983 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:54:59,983 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:54:59,983 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:54:59,983 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,993 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:54:59,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,994 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:54:59,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:54:59,994 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:54:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,995 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,995 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:54:59,996 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:54:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:54:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:54:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:54:59,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,005 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:55:00,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,005 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:55:00,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,006 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,005 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:55:00,006 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,006 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,106 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:55:00,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:55:00,116 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(933): Client=jenkins//188.40.62.62 procedure request for: flush-table-proc 2023-02-13 11:55:00,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-02-13 11:55:00,118 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-02-13 11:55:00,118 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-02-13 11:55:00,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:55:00,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-02-13 11:55:00,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-02-13 11:55:00,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,118 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,126 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,126 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,126 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,126 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,127 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,127 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-02-13 11:55:00,126 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:55:00,127 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-02-13 11:55:00,128 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:55:00,128 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:55:00,128 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44743,1676289185212' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,44985,1676289185170' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:55:00,128 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-02-13 11:55:00,129 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-02-13 11:55:00,129 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,129 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. started... 2023-02-13 11:55:00,129 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-02-13 11:55:00,136 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/b0d5cf77e7d849fd95c0766f18986805 2023-02-13 11:55:00,146 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/52289aea9c024023af5aa6ef980a4b08 2023-02-13 11:55:00,149 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/b0d5cf77e7d849fd95c0766f18986805 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805 2023-02-13 11:55:00,152 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:55:00,153 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/52289aea9c024023af5aa6ef980a4b08 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08 2023-02-13 11:55:00,154 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,154 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,155 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,155 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,155 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:55:00,155 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:55:00,156 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08, entries=10, sequenceid=71, filesize=5.0 K 2023-02-13 11:55:00,157 INFO [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2947): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 28ms, sequenceid=71, compaction requested=true 2023-02-13 11:55:00,157 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] regionserver.HRegion(2445): Flush status journal for e59db06e6e7d32a7f703cd50dccda9fd: 2023-02-13 11:55:00,157 DEBUG [rs(jenkins-hbase16.apache.org,43159,1676289185121)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,157 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-02-13 11:55:00,157 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-02-13 11:55:00,157 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,157 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-02-13 11:55:00,157 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-02-13 11:55:00,172 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,172 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:55:00,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:55:00,172 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,172 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-02-13 11:55:00,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:55:00,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:55:00,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:55:00,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase16.apache.org,43159,1676289185121' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-02-13 11:55:00,174 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-02-13 11:55:00,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76bf78b5[Count = 0] remaining members to acquire global barrier 2023-02-13 11:55:00,174 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,180 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-02-13 11:55:00,180 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44985,1676289185170' in zk 2023-02-13 11:55:00,180 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,180 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,43159,1676289185121' in zk 2023-02-13 11:55:00,181 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase16.apache.org,44743,1676289185212' in zk 2023-02-13 11:55:00,189 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:55:00,189 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:55:00,189 DEBUG [member: 'jenkins-hbase16.apache.org,44985,1676289185170' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:55:00,196 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:55:00,197 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:55:00,197 DEBUG [member: 'jenkins-hbase16.apache.org,43159,1676289185121' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:55:00,197 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:55:00,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:55:00,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:55:00,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-02-13 11:55:00,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:55:00,205 DEBUG [member: 'jenkins-hbase16.apache.org,44743,1676289185212' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-02-13 11:55:00,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:55:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,206 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,207 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase16.apache.org,43159,1676289185121': 2023-02-13 11:55:00,208 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase16.apache.org,43159,1676289185121' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-02-13 11:55:00,208 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-02-13 11:55:00,208 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-02-13 11:55:00,208 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-02-13 11:55:00,208 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-02-13 11:55:00,208 INFO [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-02-13 11:55:00,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,218 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-02-13 11:55:00,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,219 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-02-13 11:55:00,220 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-02-13 11:55:00,221 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,221 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-02-13 11:55:00,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-02-13 11:55:00,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-02-13 11:55:00,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,230 DEBUG [(jenkins-hbase16.apache.org,44445,1676289184060)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-02-13 11:55:00,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,230 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-02-13 11:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-02-13 11:55:00,230 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-02-13 11:55:00,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,231 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-02-13 11:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-02-13 11:55:00,231 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-02-13 11:55:00,331 DEBUG [Listener at localhost.localdomain/45781] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-02-13 11:55:00,332 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-02-13 11:55:00,338 DEBUG [Listener at localhost.localdomain/45781] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,338 DEBUG [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e59db06e6e7d32a7f703cd50dccda9fd, daughterA=18d604d87c45c55152ebdbaa23b68c9f, daughterB=723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:00,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e59db06e6e7d32a7f703cd50dccda9fd, daughterA=18d604d87c45c55152ebdbaa23b68c9f, daughterB=723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:00,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e59db06e6e7d32a7f703cd50dccda9fd, daughterA=18d604d87c45c55152ebdbaa23b68c9f, daughterB=723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:00,339 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e59db06e6e7d32a7f703cd50dccda9fd, daughterA=18d604d87c45c55152ebdbaa23b68c9f, daughterB=723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:00,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, UNASSIGN}] 2023-02-13 11:55:00,342 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, UNASSIGN 2023-02-13 11:55:00,342 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e59db06e6e7d32a7f703cd50dccda9fd, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,342 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289300342"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289300342"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289300342"}]},"ts":"1676289300342"} 2023-02-13 11:55:00,343 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure e59db06e6e7d32a7f703cd50dccda9fd, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:55:00,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e59db06e6e7d32a7f703cd50dccda9fd, disabling compactions & flushes 2023-02-13 11:55:00,495 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,495 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. after waiting 0 ms 2023-02-13 11:55:00,496 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,496 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing e59db06e6e7d32a7f703cd50dccda9fd 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-02-13 11:55:00,504 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/01a639f971e74ba08674f6fd5214c9dc 2023-02-13 11:55:00,513 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/97bafaab7b48415f9430949aa3ea2059 2023-02-13 11:55:00,517 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf1/01a639f971e74ba08674f6fd5214c9dc as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc 2023-02-13 11:55:00,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:55:00,522 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/.tmp/cf2/97bafaab7b48415f9430949aa3ea2059 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059 2023-02-13 11:55:00,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059, entries=8, sequenceid=82, filesize=5.0 K 2023-02-13 11:55:00,526 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 30ms, sequenceid=82, compaction requested=true 2023-02-13 11:55:00,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-02-13 11:55:00,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:55:00,530 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. 2023-02-13 11:55:00,530 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e59db06e6e7d32a7f703cd50dccda9fd: Waiting for close lock at 1676289300495Running coprocessor pre-close hooks at 1676289300495Disabling compacts and flushes for region at 1676289300495Disabling writes for close at 1676289300496 (+1 ms)Obtaining lock to block concurrent updates at 1676289300496Preparing flush snapshotting stores in e59db06e6e7d32a7f703cd50dccda9fd at 1676289300496Finished memstore snapshotting testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd., syncing WAL and waiting on mvcc, flushsize=dataSize=480, getHeapSize=2144, getOffHeapSize=0, getCellsCount=16 at 1676289300496Flushing stores of testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. at 1676289300496Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf1: creating writer at 1676289300496Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf1: appending metadata at 1676289300498 (+2 ms)Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf1: closing flushed file at 1676289300498Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf2: creating writer at 1676289300507 (+9 ms)Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf2: appending metadata at 1676289300508 (+1 ms)Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf2: closing flushed file at 1676289300508Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf1: reopening flushed file at 1676289300518 (+10 ms)Flushing e59db06e6e7d32a7f703cd50dccda9fd/cf2: reopening flushed file at 1676289300522 (+4 ms)Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e59db06e6e7d32a7f703cd50dccda9fd in 30ms, sequenceid=82, compaction requested=true at 1676289300526 (+4 ms)Writing region close event to WAL at 1676289300527 (+1 ms)Running coprocessor post-close hooks at 1676289300530 (+3 ms)Closed at 1676289300530 2023-02-13 11:55:00,531 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,532 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e59db06e6e7d32a7f703cd50dccda9fd, regionState=CLOSED 2023-02-13 11:55:00,532 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289300532"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289300532"}]},"ts":"1676289300532"} 2023-02-13 11:55:00,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-02-13 11:55:00,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure e59db06e6e7d32a7f703cd50dccda9fd, server=jenkins-hbase16.apache.org,43159,1676289185121 in 190 msec 2023-02-13 11:55:00,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-02-13 11:55:00,535 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e59db06e6e7d32a7f703cd50dccda9fd, UNASSIGN in 193 msec 2023-02-13 11:55:00,543 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=e59db06e6e7d32a7f703cd50dccda9fd, threads=14 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,543 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,544 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,548 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,688 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,723 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,732 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,733 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,739 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,756 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,764 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,768 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,772 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,773 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,773 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,775 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,775 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3 for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,775 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b for region: e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:00,775 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region e59db06e6e7d32a7f703cd50dccda9fd Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-02-13 11:55:00,816 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:55:00,817 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-02-13 11:55:00,818 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1676289300818"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1676289300818"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1676289300818"}]},"ts":"1676289300818"} 2023-02-13 11:55:00,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289300818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289300818"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289300818"}]},"ts":"1676289300818"} 2023-02-13 11:55:00,819 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289300818"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289300818"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289300818"}]},"ts":"1676289300818"} 2023-02-13 11:55:00,821 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, ASSIGN}] 2023-02-13 11:55:00,822 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, ASSIGN 2023-02-13 11:55:00,823 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:55:00,823 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, ASSIGN 2023-02-13 11:55:00,824 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase16.apache.org,43159,1676289185121; forceNewPlan=false, retain=false 2023-02-13 11:55:00,973 INFO [jenkins-hbase16:44445] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-02-13 11:55:00,975 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=18d604d87c45c55152ebdbaa23b68c9f, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,975 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289300975"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289300975"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289300975"}]},"ts":"1676289300975"} 2023-02-13 11:55:00,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 18d604d87c45c55152ebdbaa23b68c9f, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:55:00,978 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=723ab7561bcaba7014a058675ca97f33, regionState=OPENING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:00,978 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289300978"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289300978"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289300978"}]},"ts":"1676289300978"} 2023-02-13 11:55:00,979 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 723ab7561bcaba7014a058675ca97f33, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:55:01,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 723ab7561bcaba7014a058675ca97f33, NAME => 'testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.', STARTKEY => '41', ENDKEY => ''} 2023-02-13 11:55:01,132 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,132 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,134 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,135 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1 2023-02-13 11:55:01,135 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1 2023-02-13 11:55:01,135 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 723ab7561bcaba7014a058675ca97f33 columnFamilyName cf1 2023-02-13 11:55:01,152 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-top 2023-02-13 11:55:01,156 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-top 2023-02-13 11:55:01,159 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-top 2023-02-13 11:55:01,162 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-top 2023-02-13 11:55:01,166 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-top 2023-02-13 11:55:01,170 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-top 2023-02-13 11:55:01,173 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-top 2023-02-13 11:55:01,173 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(310): Store=723ab7561bcaba7014a058675ca97f33/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:55:01,173 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,174 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2 2023-02-13 11:55:01,174 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2 2023-02-13 11:55:01,174 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 723ab7561bcaba7014a058675ca97f33 columnFamilyName cf2 2023-02-13 11:55:01,185 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-top 2023-02-13 11:55:01,189 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-top 2023-02-13 11:55:01,192 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-top 2023-02-13 11:55:01,196 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-top 2023-02-13 11:55:01,200 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-top 2023-02-13 11:55:01,203 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-top 2023-02-13 11:55:01,207 DEBUG [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-top 2023-02-13 11:55:01,207 INFO [StoreOpener-723ab7561bcaba7014a058675ca97f33-1] regionserver.HStore(310): Store=723ab7561bcaba7014a058675ca97f33/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:55:01,208 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,209 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:55:01,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 723ab7561bcaba7014a058675ca97f33; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68897025, jitterRate=0.02664567530155182}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:55:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 723ab7561bcaba7014a058675ca97f33: Running coprocessor pre-open hook at 1676289301132Writing region info on filesystem at 1676289301132Initializing all the Stores at 1676289301134 (+2 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289301134Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289301134Cleaning up temporary data from old regions at 1676289301210 (+76 ms)Cleaning up detritus from prior splits at 1676289301210Running coprocessor post-open hooks at 1676289301212 (+2 ms)Region opened successfully at 1676289301212 2023-02-13 11:55:01,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33., pid=150, masterSystemTime=1676289301128 2023-02-13 11:55:01,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,213 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:55:01,213 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:55:01,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:55:01,213 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,213 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,214 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): 723ab7561bcaba7014a058675ca97f33/cf1 is initiating minor compaction (all files) 2023-02-13 11:55:01,213 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): 723ab7561bcaba7014a058675ca97f33/cf2 is initiating minor compaction (all files) 2023-02-13 11:55:01,214 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 723ab7561bcaba7014a058675ca97f33/cf1 in testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,214 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 723ab7561bcaba7014a058675ca97f33/cf2 in testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,214 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/.tmp, totalSize=34.9 K 2023-02-13 11:55:01,214 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-top] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/.tmp, totalSize=34.9 K 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289298881 2023-02-13 11:55:01,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1676289298881 2023-02-13 11:55:01,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(128): Open testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7850): Opening region: {ENCODED => 18d604d87c45c55152ebdbaa23b68c9f, NAME => 'testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.', STARTKEY => '', ENDKEY => '41'} 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289299129 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1676289299129 2023-02-13 11:55:01,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289299401 2023-02-13 11:55:01,215 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1676289299401 2023-02-13 11:55:01,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,216 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289299616 2023-02-13 11:55:01,216 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=723ab7561bcaba7014a058675ca97f33, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(865): Instantiated testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-02-13 11:55:01,216 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1676289299616 2023-02-13 11:55:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7890): checking encryption for 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,216 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301216"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289301216"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289301216"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289301216"}]},"ts":"1676289301216"} 2023-02-13 11:55:01,216 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289299887 2023-02-13 11:55:01,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(7893): checking classloading for 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,216 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1676289299887 2023-02-13 11:55:01,216 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289300108 2023-02-13 11:55:01,217 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1676289300108 2023-02-13 11:55:01,217 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289300333 2023-02-13 11:55:01,217 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1676289300333 2023-02-13 11:55:01,218 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-02-13 11:55:01,219 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1 2023-02-13 11:55:01,219 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 723ab7561bcaba7014a058675ca97f33, server=jenkins-hbase16.apache.org,43159,1676289185121 in 238 msec 2023-02-13 11:55:01,219 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1 2023-02-13 11:55:01,220 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18d604d87c45c55152ebdbaa23b68c9f columnFamilyName cf1 2023-02-13 11:55:01,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, ASSIGN in 399 msec 2023-02-13 11:55:01,231 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): 723ab7561bcaba7014a058675ca97f33#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:55:01,232 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): 723ab7561bcaba7014a058675ca97f33#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:55:01,236 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-bottom 2023-02-13 11:55:01,238 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-bottom 2023-02-13 11:55:01,242 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-bottom 2023-02-13 11:55:01,257 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-bottom 2023-02-13 11:55:01,260 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/.tmp/cf2/5c42e6b77fc945c9b33af9cbfaabb0a0 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c42e6b77fc945c9b33af9cbfaabb0a0 2023-02-13 11:55:01,265 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-bottom 2023-02-13 11:55:01,266 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/.tmp/cf1/5d954c6c29ba4c01b011e8cf1db36991 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/5d954c6c29ba4c01b011e8cf1db36991 2023-02-13 11:55:01,266 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 723ab7561bcaba7014a058675ca97f33/cf2 of 723ab7561bcaba7014a058675ca97f33 into 5c42e6b77fc945c9b33af9cbfaabb0a0(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:55:01,266 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 723ab7561bcaba7014a058675ca97f33: 2023-02-13 11:55:01,266 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33., storeName=723ab7561bcaba7014a058675ca97f33/cf2, priority=9, startTime=1676289301213; duration=0sec 2023-02-13 11:55:01,267 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,268 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-bottom 2023-02-13 11:55:01,270 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 723ab7561bcaba7014a058675ca97f33/cf1 of 723ab7561bcaba7014a058675ca97f33 into 5d954c6c29ba4c01b011e8cf1db36991(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:55:01,270 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 723ab7561bcaba7014a058675ca97f33: 2023-02-13 11:55:01,271 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33., storeName=723ab7561bcaba7014a058675ca97f33/cf1, priority=9, startTime=1676289301212; duration=0sec 2023-02-13 11:55:01,271 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,271 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-bottom 2023-02-13 11:55:01,271 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(310): Store=18d604d87c45c55152ebdbaa23b68c9f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:55:01,271 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,272 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2 2023-02-13 11:55:01,272 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2 2023-02-13 11:55:01,273 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 18d604d87c45c55152ebdbaa23b68c9f columnFamilyName cf2 2023-02-13 11:55:01,284 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-bottom 2023-02-13 11:55:01,287 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-bottom 2023-02-13 11:55:01,290 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-bottom 2023-02-13 11:55:01,293 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-bottom 2023-02-13 11:55:01,296 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-bottom 2023-02-13 11:55:01,299 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-bottom 2023-02-13 11:55:01,302 DEBUG [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(539): loaded hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-bottom 2023-02-13 11:55:01,302 INFO [StoreOpener-18d604d87c45c55152ebdbaa23b68c9f-1] regionserver.HStore(310): Store=18d604d87c45c55152ebdbaa23b68c9f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-02-13 11:55:01,303 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(5208): Found 0 recovered edits file(s) under hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-02-13 11:55:01,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1054): writing seq id for 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1071): Opened 18d604d87c45c55152ebdbaa23b68c9f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=69651033, jitterRate=0.0378812700510025}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-02-13 11:55:01,307 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(964): Region open journal for 18d604d87c45c55152ebdbaa23b68c9f: Running coprocessor pre-open hook at 1676289301216Writing region info on filesystem at 1676289301216Initializing all the Stores at 1676289301217 (+1 ms)Instantiating store for column family {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289301217Instantiating store for column family {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} at 1676289301218 (+1 ms)Cleaning up temporary data from old regions at 1676289301304 (+86 ms)Cleaning up detritus from prior splits at 1676289301305 (+1 ms)Running coprocessor post-open hooks at 1676289301307 (+2 ms)Region opened successfully at 1676289301307 2023-02-13 11:55:01,307 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2335): Post open deploy tasks for testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f., pid=149, masterSystemTime=1676289301128 2023-02-13 11:55:01,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,309 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:55:01,309 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,309 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,309 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1912): 18d604d87c45c55152ebdbaa23b68c9f/cf1 is initiating minor compaction (all files) 2023-02-13 11:55:01,309 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1912): 18d604d87c45c55152ebdbaa23b68c9f/cf2 is initiating minor compaction (all files) 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2258): Starting compaction of 18d604d87c45c55152ebdbaa23b68c9f/cf1 in testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2258): Starting compaction of 18d604d87c45c55152ebdbaa23b68c9f/cf2 in testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/.tmp, totalSize=34.9 K 2023-02-13 11:55:01,309 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-bottom] into tmpdir=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/.tmp, totalSize=34.9 K 2023-02-13 11:55:01,310 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289298881 2023-02-13 11:55:01,310 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1676289298881 2023-02-13 11:55:01,310 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionServer(2362): Finished post open deploy task for testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,310 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289299129 2023-02-13 11:55:01,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase16:0-0] handler.AssignRegionHandler(156): Opened testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,310 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1676289299129 2023-02-13 11:55:01,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=18d604d87c45c55152ebdbaa23b68c9f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289299401 2023-02-13 11:55:01,311 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301310"},{"qualifier":"server","vlen":32,"tag":[],"timestamp":"1676289301310"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1676289301310"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1676289301310"}]},"ts":"1676289301310"} 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1676289299401 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289299616 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1676289299616 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289299887 2023-02-13 11:55:01,311 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1676289299887 2023-02-13 11:55:01,312 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289300108 2023-02-13 11:55:01,312 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1676289300108 2023-02-13 11:55:01,312 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] compactions.Compactor(207): Compacting 01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289300333 2023-02-13 11:55:01,312 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] compactions.Compactor(207): Compacting 97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1676289300333 2023-02-13 11:55:01,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-02-13 11:55:01,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 18d604d87c45c55152ebdbaa23b68c9f, server=jenkins-hbase16.apache.org,43159,1676289185121 in 336 msec 2023-02-13 11:55:01,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-02-13 11:55:01,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, ASSIGN in 493 msec 2023-02-13 11:55:01,316 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=e59db06e6e7d32a7f703cd50dccda9fd, daughterA=18d604d87c45c55152ebdbaa23b68c9f, daughterB=723ab7561bcaba7014a058675ca97f33 in 977 msec 2023-02-13 11:55:01,321 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] throttle.PressureAwareThroughputController(145): 18d604d87c45c55152ebdbaa23b68c9f#cf1#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:55:01,321 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] throttle.PressureAwareThroughputController(145): 18d604d87c45c55152ebdbaa23b68c9f#cf2#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-02-13 11:55:01,336 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/.tmp/cf1/0724ad8e3eb84be0a18d3d2b9bfe845d as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/0724ad8e3eb84be0a18d3d2b9bfe845d 2023-02-13 11:55:01,337 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/.tmp/cf2/3f55ad1ee0994c37bd2f1bd9078ba112 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/3f55ad1ee0994c37bd2f1bd9078ba112 2023-02-13 11:55:01,340 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=43159] ipc.CallRunner(144): callId: 1171 service: ClientService methodName: Scan size: 143 connection: 188.40.62.62:45010 deadline: 1676289361340, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd. is not online on jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:01,342 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18d604d87c45c55152ebdbaa23b68c9f/cf2 of 18d604d87c45c55152ebdbaa23b68c9f into 3f55ad1ee0994c37bd2f1bd9078ba112(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:55:01,342 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 18d604d87c45c55152ebdbaa23b68c9f/cf1 of 18d604d87c45c55152ebdbaa23b68c9f into 0724ad8e3eb84be0a18d3d2b9bfe845d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-02-13 11:55:01,342 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.HRegion(2288): Compaction status journal for 18d604d87c45c55152ebdbaa23b68c9f: 2023-02-13 11:55:01,342 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.HRegion(2288): Compaction status journal for 18d604d87c45c55152ebdbaa23b68c9f: 2023-02-13 11:55:01,342 INFO [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f., storeName=18d604d87c45c55152ebdbaa23b68c9f/cf2, priority=9, startTime=1676289301309; duration=0sec 2023-02-13 11:55:01,342 INFO [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f., storeName=18d604d87c45c55152ebdbaa23b68c9f/cf1, priority=9, startTime=1676289301308; duration=0sec 2023-02-13 11:55:01,342 DEBUG [RS:0;jenkins-hbase16:43159-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,342 DEBUG [RS:0;jenkins-hbase16:43159-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-02-13 11:55:01,452 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:55:01,457 INFO [Listener at localhost.localdomain/45781] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-02-13 11:55:01,458 DEBUG [Listener at localhost.localdomain/45781] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-02-13 11:55:01,470 WARN [Listener at localhost.localdomain/45781] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-02-13 11:55:01,472 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-02-13 11:55:01,472 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$11(2405): Client=jenkins//188.40.62.62 disable testSplitTableRegion 2023-02-13 11:55:01,473 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-02-13 11:55:01,476 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 11:55:01,476 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289301476"}]},"ts":"1676289301476"} 2023-02-13 11:55:01,477 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-02-13 11:55:01,626 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-02-13 11:55:01,627 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 11:55:01,627 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, UNASSIGN}] 2023-02-13 11:55:01,628 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, UNASSIGN 2023-02-13 11:55:01,629 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=723ab7561bcaba7014a058675ca97f33, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:01,629 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301629"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289301629"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289301629"}]},"ts":"1676289301629"} 2023-02-13 11:55:01,630 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 723ab7561bcaba7014a058675ca97f33, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:55:01,632 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, UNASSIGN 2023-02-13 11:55:01,632 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=18d604d87c45c55152ebdbaa23b68c9f, regionState=CLOSING, regionLocation=jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:01,632 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301632"},{"qualifier":"sn","vlen":46,"tag":[],"timestamp":"1676289301632"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1676289301632"}]},"ts":"1676289301632"} 2023-02-13 11:55:01,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 18d604d87c45c55152ebdbaa23b68c9f, server=jenkins-hbase16.apache.org,43159,1676289185121}] 2023-02-13 11:55:01,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 18d604d87c45c55152ebdbaa23b68c9f, disabling compactions & flushes 2023-02-13 11:55:01,782 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. after waiting 0 ms 2023-02-13 11:55:01,782 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,783 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-bottom] to archive 2023-02-13 11:55:01,784 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:55:01,786 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,787 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,788 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,789 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,790 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,791 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,792 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,793 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-bottom, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-bottom] to archive 2023-02-13 11:55:01,793 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:55:01,794 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,795 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,796 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,797 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,798 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,798 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,799 DEBUG [StoreCloser-testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:55:01,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:55:01,802 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f. 2023-02-13 11:55:01,802 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 18d604d87c45c55152ebdbaa23b68c9f: Waiting for close lock at 1676289301782Running coprocessor pre-close hooks at 1676289301782Disabling compacts and flushes for region at 1676289301782Disabling writes for close at 1676289301782Writing region close event to WAL at 1676289301800 (+18 ms)Running coprocessor post-close hooks at 1676289301802 (+2 ms)Closed at 1676289301802 2023-02-13 11:55:01,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:01,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(109): Close 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 723ab7561bcaba7014a058675ca97f33, disabling compactions & flushes 2023-02-13 11:55:01,804 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. after waiting 0 ms 2023-02-13 11:55:01,804 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=18d604d87c45c55152ebdbaa23b68c9f, regionState=CLOSED 2023-02-13 11:55:01,804 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,804 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301804"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289301804"}]},"ts":"1676289301804"} 2023-02-13 11:55:01,805 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc-top] to archive 2023-02-13 11:55:01,805 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:55:01,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-02-13 11:55:01,806 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 18d604d87c45c55152ebdbaa23b68c9f, server=jenkins-hbase16.apache.org,43159,1676289185121 in 172 msec 2023-02-13 11:55:01,807 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/ba8acff1939a4a0c9f24628e137de6ed.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=18d604d87c45c55152ebdbaa23b68c9f, UNASSIGN in 179 msec 2023-02-13 11:55:01,807 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/1f389b847f3e440894fc7ec4afeca0e3.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,808 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/16064054a79449ce9ed8a9bf31d83e3b.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,809 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,809 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/6909e8bcb83c4e7d9ac6d4d773e90268.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,810 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/b0d5cf77e7d849fd95c0766f18986805.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,811 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/01a639f971e74ba08674f6fd5214c9dc.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,812 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08-top, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd->hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059-top] to archive 2023-02-13 11:55:01,813 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-02-13 11:55:01,814 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/f072d7aaba844c50ab63dd18af27a43e.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,815 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/404da48346b848329591df2eeb214aa8.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,815 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/261bb0f83b914a7291fe35493b0b9bf6.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,816 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/4d525582989e498d9f8a6b0fe3692f92.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,817 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c7c89c6ff724ba2a67af5d0da0add1e.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,818 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/52289aea9c024023af5aa6ef980a4b08.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,818 DEBUG [StoreCloser-testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/97bafaab7b48415f9430949aa3ea2059.e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:01,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-02-13 11:55:01,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:55:01,821 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33. 2023-02-13 11:55:01,821 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 723ab7561bcaba7014a058675ca97f33: Waiting for close lock at 1676289301804Running coprocessor pre-close hooks at 1676289301804Disabling compacts and flushes for region at 1676289301804Disabling writes for close at 1676289301804Writing region close event to WAL at 1676289301819 (+15 ms)Running coprocessor post-close hooks at 1676289301821 (+2 ms)Closed at 1676289301821 2023-02-13 11:55:01,822 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.UnassignRegionHandler(147): Closed 723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:01,822 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=723ab7561bcaba7014a058675ca97f33, regionState=CLOSED 2023-02-13 11:55:01,823 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1676289301822"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1676289301822"}]},"ts":"1676289301822"} 2023-02-13 11:55:01,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-02-13 11:55:01,824 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 723ab7561bcaba7014a058675ca97f33, server=jenkins-hbase16.apache.org,43159,1676289185121 in 193 msec 2023-02-13 11:55:01,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=153, resume processing ppid=151 2023-02-13 11:55:01,826 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=723ab7561bcaba7014a058675ca97f33, UNASSIGN in 198 msec 2023-02-13 11:55:01,826 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1676289301826"}]},"ts":"1676289301826"} 2023-02-13 11:55:01,827 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-02-13 11:55:01,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 11:55:01,834 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-02-13 11:55:01,836 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 363 msec 2023-02-13 11:55:02,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-02-13 11:55:02,129 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-02-13 11:55:02,130 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.HMaster$5(2215): Client=jenkins//188.40.62.62 delete testSplitTableRegion 2023-02-13 11:55:02,131 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,132 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,133 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 11:55:02,139 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:02,139 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:02,139 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:02,140 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/recovered.edits] 2023-02-13 11:55:02,140 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/recovered.edits] 2023-02-13 11:55:02,140 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2, FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/recovered.edits] 2023-02-13 11:55:02,142 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/0724ad8e3eb84be0a18d3d2b9bfe845d to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf1/0724ad8e3eb84be0a18d3d2b9bfe845d 2023-02-13 11:55:02,142 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/5d954c6c29ba4c01b011e8cf1db36991 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf1/5d954c6c29ba4c01b011e8cf1db36991 2023-02-13 11:55:02,143 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/01a639f971e74ba08674f6fd5214c9dc 2023-02-13 11:55:02,144 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/16064054a79449ce9ed8a9bf31d83e3b 2023-02-13 11:55:02,144 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c42e6b77fc945c9b33af9cbfaabb0a0 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/cf2/5c42e6b77fc945c9b33af9cbfaabb0a0 2023-02-13 11:55:02,144 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/3f55ad1ee0994c37bd2f1bd9078ba112 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/cf2/3f55ad1ee0994c37bd2f1bd9078ba112 2023-02-13 11:55:02,145 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/1f389b847f3e440894fc7ec4afeca0e3 2023-02-13 11:55:02,146 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/57ee0b455c3d44ebb25ce8aaed5dfc02 2023-02-13 11:55:02,146 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33/recovered.edits/90.seqid 2023-02-13 11:55:02,146 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/recovered.edits/90.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f/recovered.edits/90.seqid 2023-02-13 11:55:02,148 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/18d604d87c45c55152ebdbaa23b68c9f 2023-02-13 11:55:02,148 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/723ab7561bcaba7014a058675ca97f33 2023-02-13 11:55:02,148 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/6909e8bcb83c4e7d9ac6d4d773e90268 2023-02-13 11:55:02,149 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/b0d5cf77e7d849fd95c0766f18986805 2023-02-13 11:55:02,150 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf1/ba8acff1939a4a0c9f24628e137de6ed 2023-02-13 11:55:02,152 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/261bb0f83b914a7291fe35493b0b9bf6 2023-02-13 11:55:02,153 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/404da48346b848329591df2eeb214aa8 2023-02-13 11:55:02,154 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/4d525582989e498d9f8a6b0fe3692f92 2023-02-13 11:55:02,155 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/52289aea9c024023af5aa6ef980a4b08 2023-02-13 11:55:02,156 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/5c7c89c6ff724ba2a67af5d0da0add1e 2023-02-13 11:55:02,157 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059 to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/97bafaab7b48415f9430949aa3ea2059 2023-02-13 11:55:02,158 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/cf2/f072d7aaba844c50ab63dd18af27a43e 2023-02-13 11:55:02,160 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/recovered.edits/85.seqid to hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/archive/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd/recovered.edits/85.seqid 2023-02-13 11:55:02,160 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/.tmp/data/default/testSplitTableRegion/e59db06e6e7d32a7f703cd50dccda9fd 2023-02-13 11:55:02,161 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-02-13 11:55:02,162 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,164 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-02-13 11:55:02,165 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-02-13 11:55:02,166 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,166 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-02-13 11:55:02,166 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289302166"}]},"ts":"9223372036854775807"} 2023-02-13 11:55:02,166 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289302166"}]},"ts":"9223372036854775807"} 2023-02-13 11:55:02,166 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1676289302166"}]},"ts":"9223372036854775807"} 2023-02-13 11:55:02,168 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-02-13 11:55:02,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e59db06e6e7d32a7f703cd50dccda9fd, NAME => 'testSplitTableRegion,,1676289298377.e59db06e6e7d32a7f703cd50dccda9fd.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 18d604d87c45c55152ebdbaa23b68c9f, NAME => 'testSplitTableRegion,,1676289300338.18d604d87c45c55152ebdbaa23b68c9f.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 723ab7561bcaba7014a058675ca97f33, NAME => 'testSplitTableRegion,41,1676289300338.723ab7561bcaba7014a058675ca97f33.', STARTKEY => '41', ENDKEY => ''}] 2023-02-13 11:55:02,168 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-02-13 11:55:02,168 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1676289302168"}]},"ts":"9223372036854775807"} 2023-02-13 11:55:02,170 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-02-13 11:55:02,202 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-02-13 11:55:02,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 72 msec 2023-02-13 11:55:02,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44445] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-02-13 11:55:02,236 INFO [Listener at localhost.localdomain/45781] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-02-13 11:55:02,250 INFO [Listener at localhost.localdomain/45781] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 377) - Thread LEAK? -, OpenFileDescriptor=841 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=227 (was 229), ProcessCount=169 (was 169), AvailableMemoryMB=5209 (was 5298) 2023-02-13 11:55:02,250 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-02-13 11:55:02,250 INFO [Listener at localhost.localdomain/45781] client.ConnectionImplementation(1959): Closing master protocol: MasterService 2023-02-13 11:55:02,251 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1bf6ae7e to 127.0.0.1:55638 2023-02-13 11:55:02,251 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,252 DEBUG [Listener at localhost.localdomain/45781] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-02-13 11:55:02,252 DEBUG [Listener at localhost.localdomain/45781] util.JVMClusterUtil(257): Found active master hash=424083981, stopped=false 2023-02-13 11:55:02,252 INFO [Listener at localhost.localdomain/45781] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:55:02,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 11:55:02,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 11:55:02,259 INFO [Listener at localhost.localdomain/45781] procedure2.ProcedureExecutor(629): Stopping 2023-02-13 11:55:02,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 11:55:02,259 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-02-13 11:55:02,260 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:55:02,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:55:02,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:55:02,260 DEBUG [Listener at localhost.localdomain/45781] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x1482fe3e to 127.0.0.1:55638 2023-02-13 11:55:02,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:55:02,260 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-02-13 11:55:02,260 DEBUG [Listener at localhost.localdomain/45781] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,260 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,43159,1676289185121' ***** 2023-02-13 11:55:02,260 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 11:55:02,261 INFO [RS:0;jenkins-hbase16:43159] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 11:55:02,261 INFO [RS:0;jenkins-hbase16:43159] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 11:55:02,261 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 11:55:02,261 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,44985,1676289185170' ***** 2023-02-13 11:55:02,261 INFO [RS:0;jenkins-hbase16:43159] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 11:55:02,261 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 11:55:02,261 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:02,261 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,44743,1676289185212' ***** 2023-02-13 11:55:02,261 INFO [RS:1;jenkins-hbase16:44985] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 11:55:02,261 DEBUG [RS:0;jenkins-hbase16:43159] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0e28de38 to 127.0.0.1:55638 2023-02-13 11:55:02,261 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 11:55:02,261 INFO [RS:1;jenkins-hbase16:44985] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 11:55:02,261 INFO [Listener at localhost.localdomain/45781] regionserver.HRegionServer(2310): STOPPED: Shutdown requested 2023-02-13 11:55:02,262 INFO [RS:1;jenkins-hbase16:44985] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 11:55:02,261 DEBUG [RS:0;jenkins-hbase16:43159] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,262 INFO [RS:2;jenkins-hbase16:44743] regionserver.HeapMemoryManager(220): Stopping 2023-02-13 11:55:02,262 INFO [RS:0;jenkins-hbase16:43159] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 11:55:02,262 INFO [RS:2;jenkins-hbase16:44743] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-02-13 11:55:02,262 INFO [RS:2;jenkins-hbase16:44743] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-02-13 11:55:02,262 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:02,262 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-02-13 11:55:02,262 INFO [RS:0;jenkins-hbase16:43159] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 11:55:02,262 DEBUG [RS:1;jenkins-hbase16:44985] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x175830b3 to 127.0.0.1:55638 2023-02-13 11:55:02,262 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(3304): Received CLOSE for e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:55:02,262 DEBUG [RS:1;jenkins-hbase16:44985] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,262 INFO [RS:0;jenkins-hbase16:43159] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 11:55:02,262 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,44985,1676289185170; all regions closed. 2023-02-13 11:55:02,262 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(3304): Received CLOSE for 1588230740 2023-02-13 11:55:02,262 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:02,263 DEBUG [RS:2;jenkins-hbase16:44743] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x27b5e257 to 127.0.0.1:55638 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing e1c6db54e2dd7d1f929d02a4343c29a7, disabling compactions & flushes 2023-02-13 11:55:02,263 DEBUG [RS:2;jenkins-hbase16:44743] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1603): Closing 1588230740, disabling compactions & flushes 2023-02-13 11:55:02,263 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 11:55:02,263 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:meta,,1.1588230740 2023-02-13 11:55:02,263 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1475): Waiting on 1 regions to close 2023-02-13 11:55:02,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1625): Closing region hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:55:02,263 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1479): Online Regions={e1c6db54e2dd7d1f929d02a4343c29a7=hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7.} 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-02-13 11:55:02,263 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1479): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1646): Waiting without time limit for close lock on hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:meta,,1.1588230740 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1713): Acquired close lock on hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. after waiting 0 ms 2023-02-13 11:55:02,263 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1723): Updates disabled for region hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:55:02,263 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing 1588230740 3/3 column families, dataSize=79.49 KB heapSize=125.82 KB 2023-02-13 11:55:02,263 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2744): Flushing e1c6db54e2dd7d1f929d02a4343c29a7 1/1 column families, dataSize=78 B heapSize=488 B 2023-02-13 11:55:02,263 DEBUG [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1505): Waiting on 1588230740 2023-02-13 11:55:02,264 DEBUG [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1505): Waiting on e1c6db54e2dd7d1f929d02a4343c29a7 2023-02-13 11:55:02,285 DEBUG [RS:1;jenkins-hbase16:44985] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs 2023-02-13 11:55:02,285 INFO [RS:1;jenkins-hbase16:44985] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C44985%2C1676289185170:(num 1676289186803) 2023-02-13 11:55:02,285 DEBUG [RS:1;jenkins-hbase16:44985] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,285 INFO [RS:1;jenkins-hbase16:44985] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,286 INFO [RS:1;jenkins-hbase16:44985] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 11:55:02,286 INFO [RS:1;jenkins-hbase16:44985] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 11:55:02,286 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 11:55:02,286 INFO [RS:1;jenkins-hbase16:44985] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 11:55:02,286 INFO [RS:1;jenkins-hbase16:44985] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 11:55:02,287 INFO [RS:1;jenkins-hbase16:44985] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:44985 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44985,1676289185170 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:55:02,305 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7f19d59 rejected from java.util.concurrent.ThreadPoolExecutor@3ce097f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,305 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:55:02,305 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7c50b3ae rejected from java.util.concurrent.ThreadPoolExecutor@3ce097f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,307 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/.tmp/info/f7991a10c64149d4a6d4df41446a049e 2023-02-13 11:55:02,308 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,44985,1676289185170] 2023-02-13 11:55:02,308 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,44985,1676289185170; numProcessing=1 2023-02-13 11:55:02,308 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.37 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/info/cb56aa862c324cf599e0cb0fef106837 2023-02-13 11:55:02,312 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb56aa862c324cf599e0cb0fef106837 2023-02-13 11:55:02,315 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/.tmp/info/f7991a10c64149d4a6d4df41446a049e as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/info/f7991a10c64149d4a6d4df41446a049e 2023-02-13 11:55:02,322 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,44985,1676289185170 already deleted, retry=false 2023-02-13 11:55:02,322 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,44985,1676289185170 expired; onlineServers=2 2023-02-13 11:55:02,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/info/f7991a10c64149d4a6d4df41446a049e, entries=2, sequenceid=6, filesize=4.8 K 2023-02-13 11:55:02,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for e1c6db54e2dd7d1f929d02a4343c29a7 in 61ms, sequenceid=6, compaction requested=false 2023-02-13 11:55:02,326 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,326 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,326 INFO [regionserver/jenkins-hbase16:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,332 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/rep_barrier/9a41451c5bf84e94889509eed8d1ab96 2023-02-13 11:55:02,332 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/namespace/e1c6db54e2dd7d1f929d02a4343c29a7/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-02-13 11:55:02,333 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:55:02,334 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:55:02,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for e1c6db54e2dd7d1f929d02a4343c29a7: Waiting for close lock at 1676289302263Running coprocessor pre-close hooks at 1676289302263Disabling compacts and flushes for region at 1676289302263Disabling writes for close at 1676289302263Obtaining lock to block concurrent updates at 1676289302263Preparing flush snapshotting stores in e1c6db54e2dd7d1f929d02a4343c29a7 at 1676289302263Finished memstore snapshotting hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7., syncing WAL and waiting on mvcc, flushsize=dataSize=78, getHeapSize=472, getOffHeapSize=0, getCellsCount=2 at 1676289302263Flushing stores of hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. at 1676289302264 (+1 ms)Flushing e1c6db54e2dd7d1f929d02a4343c29a7/info: creating writer at 1676289302264Flushing e1c6db54e2dd7d1f929d02a4343c29a7/info: appending metadata at 1676289302283 (+19 ms)Flushing e1c6db54e2dd7d1f929d02a4343c29a7/info: closing flushed file at 1676289302283Flushing e1c6db54e2dd7d1f929d02a4343c29a7/info: reopening flushed file at 1676289302320 (+37 ms)Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for e1c6db54e2dd7d1f929d02a4343c29a7 in 61ms, sequenceid=6, compaction requested=false at 1676289302324 (+4 ms)Writing region close event to WAL at 1676289302329 (+5 ms)Running coprocessor post-close hooks at 1676289302333 (+4 ms)Closed at 1676289302334 (+1 ms) 2023-02-13 11:55:02,334 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1676289187164.e1c6db54e2dd7d1f929d02a4343c29a7. 2023-02-13 11:55:02,336 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9a41451c5bf84e94889509eed8d1ab96 2023-02-13 11:55:02,350 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/table/068d8e5bc59b49fa8735fd5569df89a3 2023-02-13 11:55:02,353 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 068d8e5bc59b49fa8735fd5569df89a3 2023-02-13 11:55:02,354 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/info/cb56aa862c324cf599e0cb0fef106837 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/info/cb56aa862c324cf599e0cb0fef106837 2023-02-13 11:55:02,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for cb56aa862c324cf599e0cb0fef106837 2023-02-13 11:55:02,357 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/info/cb56aa862c324cf599e0cb0fef106837, entries=59, sequenceid=210, filesize=11.1 K 2023-02-13 11:55:02,358 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/rep_barrier/9a41451c5bf84e94889509eed8d1ab96 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/rep_barrier/9a41451c5bf84e94889509eed8d1ab96 2023-02-13 11:55:02,361 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 9a41451c5bf84e94889509eed8d1ab96 2023-02-13 11:55:02,361 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/rep_barrier/9a41451c5bf84e94889509eed8d1ab96, entries=24, sequenceid=210, filesize=7.5 K 2023-02-13 11:55:02,362 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/.tmp/table/068d8e5bc59b49fa8735fd5569df89a3 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/table/068d8e5bc59b49fa8735fd5569df89a3 2023-02-13 11:55:02,365 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 068d8e5bc59b49fa8735fd5569df89a3 2023-02-13 11:55:02,366 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/table/068d8e5bc59b49fa8735fd5569df89a3, entries=36, sequenceid=210, filesize=8.0 K 2023-02-13 11:55:02,366 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(2947): Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 103ms, sequenceid=210, compaction requested=false 2023-02-13 11:55:02,370 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-02-13 11:55:02,370 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-02-13 11:55:02,371 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-02-13 11:55:02,372 INFO [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1837): Closed hbase:meta,,1.1588230740 2023-02-13 11:55:02,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] regionserver.HRegion(1557): Region close journal for 1588230740: Waiting for close lock at 1676289302263Running coprocessor pre-close hooks at 1676289302263Disabling compacts and flushes for region at 1676289302263Disabling writes for close at 1676289302263Obtaining lock to block concurrent updates at 1676289302263Preparing flush snapshotting stores in 1588230740 at 1676289302263Finished memstore snapshotting hbase:meta,,1.1588230740, syncing WAL and waiting on mvcc, flushsize=dataSize=81396, getHeapSize=128792, getOffHeapSize=0, getCellsCount=617 at 1676289302264 (+1 ms)Flushing stores of hbase:meta,,1.1588230740 at 1676289302264Flushing 1588230740/info: creating writer at 1676289302264Flushing 1588230740/info: appending metadata at 1676289302285 (+21 ms)Flushing 1588230740/info: closing flushed file at 1676289302285Flushing 1588230740/rep_barrier: creating writer at 1676289302312 (+27 ms)Flushing 1588230740/rep_barrier: appending metadata at 1676289302315 (+3 ms)Flushing 1588230740/rep_barrier: closing flushed file at 1676289302315Flushing 1588230740/table: creating writer at 1676289302337 (+22 ms)Flushing 1588230740/table: appending metadata at 1676289302339 (+2 ms)Flushing 1588230740/table: closing flushed file at 1676289302339Flushing 1588230740/info: reopening flushed file at 1676289302354 (+15 ms)Flushing 1588230740/rep_barrier: reopening flushed file at 1676289302358 (+4 ms)Flushing 1588230740/table: reopening flushed file at 1676289302362 (+4 ms)Finished flush of dataSize ~79.49 KB/81396, heapSize ~125.77 KB/128792, currentSize=0 B/0 for 1588230740 in 103ms, sequenceid=210, compaction requested=false at 1676289302366 (+4 ms)Writing region close event to WAL at 1676289302368 (+2 ms)Running coprocessor post-close hooks at 1676289302370 (+2 ms)Closed at 1676289302372 (+2 ms) 2023-02-13 11:55:02,372 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase16:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-02-13 11:55:02,414 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,414 INFO [RS:1;jenkins-hbase16:44985] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,44985,1676289185170; zookeeper connection closed. 2023-02-13 11:55:02,414 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@436884e rejected from java.util.concurrent.ThreadPoolExecutor@3ce097f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,414 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44985-0x10052bc7cb00002, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,414 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@3ea2888d] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@3ea2888d 2023-02-13 11:55:02,414 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@5fd7629c rejected from java.util.concurrent.ThreadPoolExecutor@3ce097f[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 310] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,464 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,43159,1676289185121; all regions closed. 2023-02-13 11:55:02,464 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,44743,1676289185212; all regions closed. 2023-02-13 11:55:02,473 DEBUG [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs 2023-02-13 11:55:02,473 INFO [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C43159%2C1676289185121.meta:.meta(num 1676289186887) 2023-02-13 11:55:02,477 DEBUG [RS:2;jenkins-hbase16:44743] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs 2023-02-13 11:55:02,477 INFO [RS:2;jenkins-hbase16:44743] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C44743%2C1676289185212:(num 1676289186803) 2023-02-13 11:55:02,477 DEBUG [RS:2;jenkins-hbase16:44743] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,477 INFO [RS:2;jenkins-hbase16:44743] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,477 INFO [RS:2;jenkins-hbase16:44743] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 11:55:02,478 INFO [RS:2;jenkins-hbase16:44743] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-02-13 11:55:02,478 INFO [RS:2;jenkins-hbase16:44743] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-02-13 11:55:02,478 INFO [RS:2;jenkins-hbase16:44743] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-02-13 11:55:02,478 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 11:55:02,478 INFO [RS:2;jenkins-hbase16:44743] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:44743 2023-02-13 11:55:02,479 DEBUG [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(932): Moved 1 WAL file(s) to /user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/oldWALs 2023-02-13 11:55:02,479 INFO [RS:0;jenkins-hbase16:43159] wal.AbstractFSWAL(935): Closed WAL: AsyncFSWAL jenkins-hbase16.apache.org%2C43159%2C1676289185121:(num 1676289186803) 2023-02-13 11:55:02,479 DEBUG [RS:0;jenkins-hbase16:43159] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,479 INFO [RS:0;jenkins-hbase16:43159] regionserver.LeaseManager(133): Closed leases 2023-02-13 11:55:02,480 INFO [RS:0;jenkins-hbase16:43159] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase16:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-02-13 11:55:02,480 INFO [regionserver/jenkins-hbase16:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 11:55:02,481 INFO [RS:0;jenkins-hbase16:43159] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:43159 2023-02-13 11:55:02,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-02-13 11:55:02,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:02,488 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,44743,1676289185212 2023-02-13 11:55:02,488 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@7d986aa3 rejected from java.util.concurrent.ThreadPoolExecutor@79a7f1c8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,497 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:02,497 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase16.apache.org,43159,1676289185121 2023-02-13 11:55:02,497 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@46a6c4ad rejected from java.util.concurrent.ThreadPoolExecutor@429958a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,497 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@33a0a7ee rejected from java.util.concurrent.ThreadPoolExecutor@79a7f1c8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,505 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,44743,1676289185212] 2023-02-13 11:55:02,505 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,44743,1676289185212; numProcessing=2 2023-02-13 11:55:02,522 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,44743,1676289185212 already deleted, retry=false 2023-02-13 11:55:02,522 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,44743,1676289185212 expired; onlineServers=1 2023-02-13 11:55:02,522 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase16.apache.org,43159,1676289185121] 2023-02-13 11:55:02,522 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase16.apache.org,43159,1676289185121; numProcessing=3 2023-02-13 11:55:02,530 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase16.apache.org,43159,1676289185121 already deleted, retry=false 2023-02-13 11:55:02,530 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase16.apache.org,43159,1676289185121 expired; onlineServers=0 2023-02-13 11:55:02,530 INFO [RegionServerTracker-0] regionserver.HRegionServer(2296): ***** STOPPING region server 'jenkins-hbase16.apache.org,44445,1676289184060' ***** 2023-02-13 11:55:02,530 INFO [RegionServerTracker-0] regionserver.HRegionServer(2310): STOPPED: Cluster shutdown set; onlineServer=0 2023-02-13 11:55:02,531 DEBUG [M:0;jenkins-hbase16:44445] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@587a0836, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase16.apache.org/188.40.62.62:0 2023-02-13 11:55:02,531 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegionServer(1145): stopping server jenkins-hbase16.apache.org,44445,1676289184060 2023-02-13 11:55:02,531 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegionServer(1171): stopping server jenkins-hbase16.apache.org,44445,1676289184060; all regions closed. 2023-02-13 11:55:02,531 DEBUG [M:0;jenkins-hbase16:44445] ipc.AbstractRpcClient(495): Stopping rpc client 2023-02-13 11:55:02,531 DEBUG [M:0;jenkins-hbase16:44445] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-02-13 11:55:02,531 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-02-13 11:55:02,531 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676289186352] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.small.0-1676289186352,5,FailOnTimeoutGroup] 2023-02-13 11:55:02,531 DEBUG [master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676289186352] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase16:0:becomeActiveMaster-HFileCleaner.large.0-1676289186352,5,FailOnTimeoutGroup] 2023-02-13 11:55:02,531 DEBUG [M:0;jenkins-hbase16:44445] cleaner.HFileCleaner(317): Stopping file delete threads 2023-02-13 11:55:02,532 INFO [M:0;jenkins-hbase16:44445] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-02-13 11:55:02,532 INFO [M:0;jenkins-hbase16:44445] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-02-13 11:55:02,532 INFO [M:0;jenkins-hbase16:44445] hbase.ChoreService(369): Chore service for: master/jenkins-hbase16:0 had [] on shutdown 2023-02-13 11:55:02,532 DEBUG [M:0;jenkins-hbase16:44445] master.HMaster(1502): Stopping service threads 2023-02-13 11:55:02,532 INFO [M:0;jenkins-hbase16:44445] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-02-13 11:55:02,532 ERROR [M:0;jenkins-hbase16:44445] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-02-13 11:55:02,532 INFO [M:0;jenkins-hbase16:44445] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-02-13 11:55:02,532 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-02-13 11:55:02,538 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-02-13 11:55:02,538 DEBUG [M:0;jenkins-hbase16:44445] zookeeper.ZKUtil(398): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-02-13 11:55:02,538 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-02-13 11:55:02,538 WARN [M:0;jenkins-hbase16:44445] master.ActiveMasterManager(323): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-02-13 11:55:02,538 INFO [M:0;jenkins-hbase16:44445] assignment.AssignmentManager(315): Stopping assignment manager 2023-02-13 11:55:02,539 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-02-13 11:55:02,539 INFO [M:0;jenkins-hbase16:44445] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-02-13 11:55:02,539 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegion(1603): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-02-13 11:55:02,539 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegion(1625): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:55:02,539 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegion(1646): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:55:02,539 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegion(1713): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-02-13 11:55:02,539 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegion(1723): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:55:02,539 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegion(2744): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=608.80 KB heapSize=719.66 KB 2023-02-13 11:55:02,549 INFO [M:0;jenkins-hbase16:44445] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=608.80 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3c04115884674d71a1ab220bf2fa59d3 2023-02-13 11:55:02,553 INFO [M:0;jenkins-hbase16:44445] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3c04115884674d71a1ab220bf2fa59d3 2023-02-13 11:55:02,554 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegionFileSystem(485): Committing hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/3c04115884674d71a1ab220bf2fa59d3 as hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3c04115884674d71a1ab220bf2fa59d3 2023-02-13 11:55:02,557 INFO [M:0;jenkins-hbase16:44445] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 3c04115884674d71a1ab220bf2fa59d3 2023-02-13 11:55:02,557 INFO [M:0;jenkins-hbase16:44445] regionserver.HStore(1080): Added hdfs://localhost.localdomain:44261/user/jenkins/test-data/a6ed4ce9-9404-ea36-5fd0-ba14e4dc9567/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/3c04115884674d71a1ab220bf2fa59d3, entries=156, sequenceid=1282, filesize=34.9 K 2023-02-13 11:55:02,558 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegion(2947): Finished flush of dataSize ~608.80 KB/623410, heapSize ~719.65 KB/736920, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1282, compaction requested=false 2023-02-13 11:55:02,558 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegion(1837): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-02-13 11:55:02,558 DEBUG [M:0;jenkins-hbase16:44445] regionserver.HRegion(1557): Region close journal for 1595e783b53d99cd5eef43b6debb2682: Waiting for close lock at 1676289302539Disabling compacts and flushes for region at 1676289302539Disabling writes for close at 1676289302539Obtaining lock to block concurrent updates at 1676289302539Preparing flush snapshotting stores in 1595e783b53d99cd5eef43b6debb2682 at 1676289302539Finished memstore snapshotting master:store,,1.1595e783b53d99cd5eef43b6debb2682., syncing WAL and waiting on mvcc, flushsize=dataSize=623410, getHeapSize=736920, getOffHeapSize=0, getCellsCount=1499 at 1676289302539Flushing stores of master:store,,1.1595e783b53d99cd5eef43b6debb2682. at 1676289302540 (+1 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: creating writer at 1676289302540Flushing 1595e783b53d99cd5eef43b6debb2682/proc: appending metadata at 1676289302543 (+3 ms)Flushing 1595e783b53d99cd5eef43b6debb2682/proc: closing flushed file at 1676289302543Flushing 1595e783b53d99cd5eef43b6debb2682/proc: reopening flushed file at 1676289302554 (+11 ms)Finished flush of dataSize ~608.80 KB/623410, heapSize ~719.65 KB/736920, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 19ms, sequenceid=1282, compaction requested=false at 1676289302558 (+4 ms)Writing region close event to WAL at 1676289302558Closed at 1676289302558 2023-02-13 11:55:02,562 INFO [M:0;jenkins-hbase16:44445] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-02-13 11:55:02,562 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-02-13 11:55:02,562 INFO [M:0;jenkins-hbase16:44445] ipc.NettyRpcServer(158): Stopping server on /188.40.62.62:44445 2023-02-13 11:55:02,571 DEBUG [M:0;jenkins-hbase16:44445] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase16.apache.org,44445,1676289184060 already deleted, retry=false 2023-02-13 11:55:02,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,605 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@4bbc8798 rejected from java.util.concurrent.ThreadPoolExecutor@79a7f1c8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,605 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:44743-0x10052bc7cb00003, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,605 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@6a23dc7f rejected from java.util.concurrent.ThreadPoolExecutor@79a7f1c8[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 312] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,605 INFO [RS:2;jenkins-hbase16:44743] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,44743,1676289185212; zookeeper connection closed. 2023-02-13 11:55:02,606 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@5c95c8ba] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@5c95c8ba 2023-02-13 11:55:02,652 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-02-13 11:55:02,705 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,706 INFO [M:0;jenkins-hbase16:44445] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,44445,1676289184060; zookeeper connection closed. 2023-02-13 11:55:02,706 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@1ed38348 rejected from java.util.concurrent.ThreadPoolExecutor@53df1761[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,706 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): master:44445-0x10052bc7cb00000, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,706 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@61df8c4d rejected from java.util.concurrent.ThreadPoolExecutor@53df1761[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 752] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,806 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,806 INFO [RS:0;jenkins-hbase16:43159] regionserver.HRegionServer(1228): Exiting; stopping=jenkins-hbase16.apache.org,43159,1676289185121; zookeeper connection closed. 2023-02-13 11:55:02,806 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@46259a0f rejected from java.util.concurrent.ThreadPoolExecutor@429958a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,806 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@2886f494] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@2886f494 2023-02-13 11:55:02,806 DEBUG [Listener at localhost.localdomain/45781-EventThread] zookeeper.ZKWatcher(600): regionserver:43159-0x10052bc7cb00001, quorum=127.0.0.1:55638, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-02-13 11:55:02,806 ERROR [Listener at localhost.localdomain/45781-EventThread] zookeeper.ClientCnxn$EventThread(537): Error while calling watcher java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@3e02a7f1 rejected from java.util.concurrent.ThreadPoolExecutor@429958a1[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 313] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379) at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112) at java.util.concurrent.Executors$DelegatedExecutorService.submit(Executors.java:678) at org.apache.hadoop.hbase.zookeeper.ZKWatcher.process(ZKWatcher.java:602) at org.apache.hadoop.hbase.zookeeper.PendingWatcher.process(PendingWatcher.java:38) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:535) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:510) 2023-02-13 11:55:02,807 INFO [Listener at localhost.localdomain/45781] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-02-13 11:55:02,807 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 11:55:02,836 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 11:55:02,941 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 11:55:02,941 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid 1e23576d-1428-4e1d-8a86-c9c4402d4f5d) service to localhost.localdomain/127.0.0.1:44261 2023-02-13 11:55:02,944 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data5/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:02,944 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data6/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:02,947 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 11:55:02,949 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 11:55:03,055 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 11:55:03,055 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid a211374e-09f0-4304-98b9-bf41d73f5c15) service to localhost.localdomain/127.0.0.1:44261 2023-02-13 11:55:03,056 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data3/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:03,056 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data4/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:03,057 WARN [Listener at localhost.localdomain/45781] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-02-13 11:55:03,059 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-02-13 11:55:03,164 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-02-13 11:55:03,164 WARN [BP-711776720-188.40.62.62-1676289179769 heartbeating to localhost.localdomain/127.0.0.1:44261] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-711776720-188.40.62.62-1676289179769 (Datanode Uuid 94b0e9f0-f834-4d10-b805-49b63beb355b) service to localhost.localdomain/127.0.0.1:44261 2023-02-13 11:55:03,165 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data1/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:03,165 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/aef81278-1029-aea4-1e1e-e627f6e187ed/cluster_a4315774-bbc2-daff-cd2e-92a5d3b46307/dfs/data/data2/current/BP-711776720-188.40.62.62-1676289179769] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-02-13 11:55:03,185 INFO [Listener at localhost.localdomain/45781] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost.localdomain:0 2023-02-13 11:55:03,362 INFO [Listener at localhost.localdomain/45781] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-02-13 11:55:03,590 INFO [Listener at localhost.localdomain/45781] hbase.HBaseTestingUtility(1293): Minicluster is down